From nnorwitz at gmail.com  Tue Apr  1 06:34:10 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 31 Mar 2008 21:34:10 -0700
Subject: [Python-Dev] refleaks on trunk
Message-ID: <ee2a432c0803312134y3a121331j2ff4e266cc634a56@mail.gmail.com>

test_io is the only leaky test on trunk that I know of.  I narrowed
down the leaks to the code below.

It's possible there are other leaks in test_io.

n
--
import sys, gc
import _fileio, io

class FileIO(_fileio._FileIO, io.RawIOBase):
    def close(self):
        io.RawIOBase.close(self)

def main():
  class MyFileIO(FileIO): pass
  f = MyFileIO('tt-test', "w")
  f.close()

for i in range(10):
  main()
  print(gc.collect())
  print(sys.gettotalrefcount())

From nnorwitz at gmail.com  Tue Apr  1 08:42:36 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 31 Mar 2008 23:42:36 -0700
Subject: [Python-Dev] fixing tests on windows
Message-ID: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>

The Windows buildbots are still failing because some tests keep files
opened.  This causes subsequent tests which use the same file to fail.

Here is a recent run which had a failure early on:
http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1209/step-test/0

I'm assuming the first failure (test_bufio) was due to an open file.
(Can't tell, no error msg.)  That means the problem was in that test
or an earlier test.  The only earlier tests are:

test_import
test_sys
test_descr
test_xdrlib
test_urllibnet
test_binhex
test_strptime
test_importhooks
test_copy
test_hmac
test_genericpath
test_complex
test_timeout
test_quopri
test_marshal
test_zipfile
test_mutants
test_csv
test_ucn
test_hash
test_wsgiref
test_mmap
test_ftplib
test_pickletools
test_codecmaps_cn

Of those tests, only some of them reference TESTFN which is the probable file:

test_import 22
test_descr 3
test_urllibnet 2
test_binhex 2
test_genericpath 44
test_complex 3
test_marshal 33
test_zipfile 136
test_mutants 7
test_mmap 32

I inspected test_mmap which had been modified somewhat recently and
tried to fix a few things.  I'm not sure I helped and there are
probably more problems lurking.

This issue may not be noticed in the default test run.  It definitely
occurs when the tests are run in a random order as they are on the
buildbots.  Try running:  ./python.exe ./Lib/test/regrtest.py -r

That should help provoke more errors.

As always, patches are greatly appreciated.

Cheers,
n

From amauryfa at gmail.com  Tue Apr  1 09:37:12 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 1 Apr 2008 09:37:12 +0200
Subject: [Python-Dev] refleaks on trunk
In-Reply-To: <ee2a432c0803312134y3a121331j2ff4e266cc634a56@mail.gmail.com>
References: <ee2a432c0803312134y3a121331j2ff4e266cc634a56@mail.gmail.com>
Message-ID: <e27efe130804010037u63afc1c9ne88b23f4d4390b08@mail.gmail.com>

On Tue, Apr 1, 2008 at 6:34 AM, Neal Norwitz <nnorwitz at gmail.com> wrote:
> test_io is the only leaky test on trunk that I know of.  I narrowed
>  down the leaks to the code below.
>
>  It's possible there are other leaks in test_io.
>
>  n
>  --
>  import sys, gc
>  import _fileio, io
>
>  class FileIO(_fileio._FileIO, io.RawIOBase):
>     def close(self):
>         io.RawIOBase.close(self)
>
>  def main():
>   class MyFileIO(FileIO): pass
>   f = MyFileIO('tt-test', "w")
>   f.close()
>
>  for i in range(10):
>   main()
>   print(gc.collect())
>   print(sys.gettotalrefcount())

The problem is that the MyFileIO class is registered in
io.RawIOBase._abc_cache, and never freed.
This is a problem with ABCs: _abc_cache should be changed to a
WeakSet, like python3.0 did.

I filed http://bugs.python.org/issue2521 about this problem.

-- 
Amaury Forgeot d'Arc

From mhammond at skippinet.com.au  Tue Apr  1 11:45:57 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue, 1 Apr 2008 20:45:57 +1100
Subject: [Python-Dev] FW: [issue2513] 64bit cross compilation on windows
In-Reply-To: <002401c892ff$a8254630$f86fd290$@com.au>
References: <002401c892ff$a8254630$f86fd290$@com.au>
Message-ID: <00c801c893dd$32d03320$98709960$@com.au>

I wrote:

> FYI, I've uploaded a patch that provides for cross-compilation on
> Windows between 32 and 64 bit platforms - all comments invited!

While I have some people's attention I'd like to re-raise another issue I
foresee for x64 builds.  I've mentioned this over the last couple of months,
but haven't got much of a response, so this seems like a reasonable
opportunity to try one more time :)
 
Currently, the "official" (by way of being de-facto) directory structure for
a build tree is 'PCBuild/.' for x86 builds and 'PCBuild/amd64' for x64
platforms.  I believe this might cause problems for people trying to port
their applications to 64bit platforms.  My proposal is that we change this
subtly - that both 'PCBuild/x86', and 'PCBuild/amd64' exist, while
'PCBuild/.' is always the 'native' platform - ie, that 'PCBuild/.' be a copy
of one of the platform subdirectories.

The rationale is fairly simple: I'm quite certain that this new directory
layout will break existing "native only" build processes (ie, those that
aren't aware of cross-compilation).  I'm quite certain that Mozilla will
break, for example, and no cross-compilation process exists that I am aware
of.  Existing build tools already know about the PCBuild directory, and are
focused almost exclusively towards native compilation - all such tools on
non-x86 platforms are currently broken.  My proposal would help avoid
breakage for existing build processes or tools that try to natively target
x64, and give a reasonable story for build processes that explicitly support
cross-compilation.  I believe the costs are fairly small - disk space is
cheap and the extra complexity in the VS project files should be reasonable.

What do people think?  I think it's time to take the approach of "silent
ascent", so unless I hear objections I'll upload a new patch which
implements this and also takes it into account for Distutils builds.

Cheers, 

Mark



From steven.bethard at gmail.com  Tue Apr  1 17:00:26 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 1 Apr 2008 09:00:26 -0600
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
Message-ID: <d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>

On Tue, Apr 1, 2008 at 12:42 AM, Neal Norwitz <nnorwitz at gmail.com> wrote:
> The Windows buildbots are still failing because some tests keep files
>  opened.  This causes subsequent tests which use the same file to fail.
>
>  Here is a recent run which had a failure early on:
>  http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1209/step-test/0
>
>  I'm assuming the first failure (test_bufio) was due to an open file.
>  (Can't tell, no error msg.)  That means the problem was in that test
>  or an earlier test.  The only earlier tests are:
>
>  test_import
>  test_sys
>  test_descr
>  test_xdrlib
>  test_urllibnet
>  test_binhex
>  test_strptime
>  test_importhooks
>  test_copy
>  test_hmac
>  test_genericpath
>  test_complex
>  test_timeout
>  test_quopri
>  test_marshal
>  test_zipfile
>  test_mutants
>  test_csv
>  test_ucn
>  test_hash
>  test_wsgiref
>  test_mmap
>  test_ftplib
>  test_pickletools
>  test_codecmaps_cn
>
>  Of those tests, only some of them reference TESTFN which is the probable file:
>
>  test_import 22
>  test_descr 3
>  test_urllibnet 2
>  test_binhex 2
>  test_genericpath 44
>  test_complex 3
>  test_marshal 33
>  test_zipfile 136
>  test_mutants 7
>  test_mmap 32
>
>  I inspected test_mmap which had been modified somewhat recently and
>  tried to fix a few things.  I'm not sure I helped and there are
>  probably more problems lurking.
>
>  This issue may not be noticed in the default test run.  It definitely
>  occurs when the tests are run in a random order as they are on the
>  buildbots.  Try running:  ./python.exe ./Lib/test/regrtest.py -r
>
>  That should help provoke more errors.

At the sprints, I ran into a bunch of similar errors running the test
suite on my Windows Vista box, even on tests that were properly
cleaning up after themselves in tearDown(). I even tried putting in
sleeps as long as 1 second, to no avail. The only way to get the test
suite to run without these errors was to stop the Windows Search
Service, fully disable Icon Overlays for TortoiseSVN, and then close
down all open folders.

Any chance the boxes the tests are being run on are running the
Windows Search Service or have Icon Overlays enabled for TortoiseSVN?

(If anyone has any ideas of how to get around these problems, I'd love
to hear them. Seems like I shouldn't have to disable these services.)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From skip at pobox.com  Tue Apr  1 17:05:28 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 1 Apr 2008 10:05:28 -0500
Subject: [Python-Dev] test_signal on osx g4
Message-ID: <18418.20280.404970.167329@montanaro-dyndns-org.local>

test_signal is failing on osx g4:

    test test_signal failed -- Traceback (most recent call last):
      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 151, in test_main
        self.fail(tb)
    AssertionError: Traceback (most recent call last):
      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 134, in test_main
        self.run_test()
      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 80, in run_test
        child = subprocess.Popen(['kill', '-HUP', str(pid)])
      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py", line 593, in __init__
        errread, errwrite)
      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py", line 1078, in _execute_child
        data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
    OSError: [Errno 4] Interrupted system call

This is the code which reads stderr from the child process:

    data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
    os.close(errpipe_read)
    if data != "":
        os.waitpid(self.pid, 0)
        child_exception = pickle.loads(data)
        raise child_exception

I'm not an expert in this stuff my any stretch of the imagination, but
shouldn't subprocess try harder to read this output?  Something like:

    while True:
        try:
            data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
        except OSError, err:
            if err.errno == errno.EINTR:
                continue
            else:
                raise
        else:
            os.close(errpipe_read)
            if data != "":
                os.waitpid(self.pid, 0)
                child_exception = pickle.loads(data)
                raise child_exception
            break

Maybe not while True, but try a few times at least.

Or is the system supposed to automatically restart interrupted system calls?

Skip

From mail at timgolden.me.uk  Tue Apr  1 17:20:46 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 01 Apr 2008 16:20:46 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
Message-ID: <47F252CE.1020702@timgolden.me.uk>

Steven Bethard wrote:
> At the sprints, I ran into a bunch of similar errors running the test
> suite on my Windows Vista box, even on tests that were properly
> cleaning up after themselves in tearDown(). I even tried putting in
> sleeps as long as 1 second, to no avail. The only way to get the test
> suite to run without these errors was to stop the Windows Search
> Service, fully disable Icon Overlays for TortoiseSVN, and then close
> down all open folders.
> 
> Any chance the boxes the tests are being run on are running the
> Windows Search Service or have Icon Overlays enabled for TortoiseSVN?
> 
> (If anyone has any ideas of how to get around these problems, I'd love
> to hear them. Seems like I shouldn't have to disable these services.)

I'm not sure I'm going to help here, but the reason that
this happens is that certain services -- and the Desktop
Search tools are notorious for this -- get file handles with
FILE_SHARE_DELETE, which means that the file isn't actually
removed when it's deleted; only when the last of those handles
closes.

I did see a suggestion somewhere that, to circumvent this issue,
instead of removing it a file you need to move it to %TEMP%
(or wherever) and delete it there. I'll try to knock up a test
case to see if this works.

TJG

From mail at timgolden.me.uk  Tue Apr  1 18:13:34 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 01 Apr 2008 17:13:34 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F252CE.1020702@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>
Message-ID: <47F25F2E.4070408@timgolden.me.uk>

Tim Golden wrote:
> Steven Bethard wrote:
>> At the sprints, I ran into a bunch of similar errors running the test
>> suite on my Windows Vista box, even on tests that were properly
>> cleaning up after themselves in tearDown(). I even tried putting in
>> sleeps as long as 1 second, to no avail. The only way to get the test
>> suite to run without these errors was to stop the Windows Search
>> Service, fully disable Icon Overlays for TortoiseSVN, and then close
>> down all open folders.
>>
>> Any chance the boxes the tests are being run on are running the
>> Windows Search Service or have Icon Overlays enabled for TortoiseSVN?
>>
>> (If anyone has any ideas of how to get around these problems, I'd love
>> to hear them. Seems like I shouldn't have to disable these services.)
> 
> I'm not sure I'm going to help here, but the reason that
> this happens is that certain services -- and the Desktop
> Search tools are notorious for this -- get file handles with
> FILE_SHARE_DELETE, which means that the file isn't actually
> removed when it's deleted; only when the last of those handles
> closes.
> 
> I did see a suggestion somewhere that, to circumvent this issue,
> instead of removing it a file you need to move it to %TEMP%
> (or wherever) and delete it there. I'll try to knock up a test
> case to see if this works.

OK: to confirm, the following test seems to indicate that
doing a (local - same volume) rename followed by a remove
will do the right thing, even when a FILE_SHARE_DELETE handle
is held.

If this is the thing to do, presumably test_support should
grow a "remove_file" which does something of this sort?

TJG

<code>
import os, sys
import win32file

FILENAME = "test"

def rename_and_remove (filename):
   os.rename (filename, filename + ".deleted")
   os.remove (filename + ".deleted")

def remove_only (filename):
   os.remove (filename)

def test (remove):
   open (FILENAME, "w").close ()
   hFile = win32file.CreateFile (
     FILENAME,
     win32file.GENERIC_READ, win32file.FILE_SHARE_DELETE,
     None, win32file.OPEN_EXISTING, 0, 0
   )
   try:
     remove (FILENAME)
     try:
       open (FILENAME, "w").close ()
     except IOError:
       print "Couldn't recreate"
     else:
       print "Could recreate"
   finally:
     hFile.Close ()

   try:
     open (FILENAME, "w").close ()
   except IOError:
     print "Couldn't recreate"
   else:
     print "Could recreate"

if __name__ =='__main__':
   print
   print "Should not work"
   test (remove_only)

   print
   print "Should work"
   test (rename_and_remove)

</code>


From facundobatista at gmail.com  Tue Apr  1 18:20:10 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 1 Apr 2008 13:20:10 -0300
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F25F2E.4070408@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
Message-ID: <e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>

2008/4/1, Tim Golden <mail at timgolden.me.uk>:

>  If this is the thing to do, presumably test_support should
>  grow a "remove_file" which does something of this sort?

+1 (I was thinking exactly that).

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From steven.bethard at gmail.com  Tue Apr  1 18:27:49 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 1 Apr 2008 10:27:49 -0600
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>
Message-ID: <d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>

On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
<facundobatista at gmail.com> wrote:
> 2008/4/1, Tim Golden <mail at timgolden.me.uk>:
>  >  If this is the thing to do, presumably test_support should
>  >  grow a "remove_file" which does something of this sort?
>
>  +1 (I was thinking exactly that).

+1 here too. That looks like a great solution.  Of course, once it's
in test_support, we need to fix *all* file removals in the test suite.
;-)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From sesquile at gmail.com  Tue Apr  1 21:02:13 2008
From: sesquile at gmail.com (m h)
Date: Tue, 1 Apr 2008 13:02:13 -0600
Subject: [Python-Dev] Tracing at a more granular level (branch coverage)
Message-ID: <e36b84ee0804011202j374d0fa1j5fb8d654eb8b10e@mail.gmail.com>

Howdy Folks-

I hope this is not too off topic.  Sadly the current sys.settrace only
allows tracing at the line level.  This isn't sufficient to do branch
and path coverage.  The main problem is that many boolean operations
can appear on a single line, and so you can't be sure which
conditional branch was taken using the current tracing method.

After speaking about code complexity/testing at Pycon [0], with a few
people there, and seeing a few people have made some strides or shown
interested in metrics/code graphs/flow and branch coverage I think
that there is interest in this.

One proposed method of getting branch coverage was to use Dalke's
Python4Ply [1] to translate code so that all branches occur on their
own line.  Then using line coverage on that and converting it back to
branch coverage of the original code.

The maintainer of coverage.py suggested that we look into patching
python instead to trace at a more granular level.  His feeling was
that there would be too many corner cases and the translation would
get hairy quite quickly.

Sadly in my 8 years of python experience I've yet to touch any c based
guts of python.  I'm looking for advice on how to get finer grain
tracing with sys.settrace.

Any advice or suggestions?  There is a quorum of people (at least 5
others) who would be very interested in this functionality because it
could lead to some cool tools built on top of it.  (With the end goal
that python code be cleaner, simpler and better tested).

thanks much,

-matt harrison


0 - http://panela.blog-city.com/pycon_2008_managing_complexity_slides.htm
1 - http://www.dalkescientific.com/Python/python4ply.html

From gnewsg at gmail.com  Tue Apr  1 21:03:54 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Tue, 1 Apr 2008 12:03:54 -0700 (PDT)
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com> 
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com> 
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk> 
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com> 
	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>
Message-ID: <02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>



On 1 Apr, 18:27, "Steven Bethard" <steven.beth... at gmail.com> wrote:
> On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
>
> <facundobati... at gmail.com> wrote:
> > 2008/4/1, Tim Golden <m... at timgolden.me.uk>:
> > ?> ?If this is the thing to do, presumably test_support should
> > ?> ?grow a "remove_file" which does something of this sort?
>
> > ?+1 (I was thinking exactly that).
>
> +1 here too. That looks like a great solution. ?Of course, once it's
> in test_support, we need to fix *all* file removals in the test suite.
> ;-)
>
> Steve

Why not just modifying test_support.unlink() like this?
Surely more convenient than modifying the whole suite.


def unlink(filename):
    try:
        if os.name == 'nt':
            os.rename(filename, filename + ".deleted")
            filename = filename + ".deleted"
        os.unlink(filename)
    except OSError:
        pass

From mail at timgolden.me.uk  Tue Apr  1 21:06:11 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 01 Apr 2008 20:06:11 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>
	<47F25F2E.4070408@timgolden.me.uk>
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>
	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>
	<02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
Message-ID: <47F287A3.3090703@timgolden.me.uk>

Giampaolo Rodola' wrote:
> 
> On 1 Apr, 18:27, "Steven Bethard" <steven.beth... at gmail.com> wrote:
>> On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
>>
>> <facundobati... at gmail.com> wrote:
>>> 2008/4/1, Tim Golden <m... at timgolden.me.uk>:
>>>  >  If this is the thing to do, presumably test_support should
>>>  >  grow a "remove_file" which does something of this sort?
>>>  +1 (I was thinking exactly that).
>> +1 here too. That looks like a great solution.  Of course, once it's
>> in test_support, we need to fix *all* file removals in the test suite.
>> ;-)
>>
>> Steve
> 
> Why not just modifying test_support.unlink() like this?
> Surely more convenient than modifying the whole suite.
> 
> 
> def unlink(filename):
>     try:
>         if os.name == 'nt':
>             os.rename(filename, filename + ".deleted")
>             filename = filename + ".deleted"
>         os.unlink(filename)
>     except OSError:
>         pass

Funnily enough, that's exactly what the patch I've
put together does. Have to look out, though for
anywhere which just calls os.unlink rather than
going through test_support.

TJG

From gnewsg at gmail.com  Tue Apr  1 21:09:56 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Tue, 1 Apr 2008 12:09:56 -0700 (PDT)
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com> 
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com> 
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk> 
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com> 
	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com> 
	<02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
Message-ID: <aa04a201-49c4-43b0-b62c-8201fd95f2c6@s37g2000prg.googlegroups.com>

On 1 Apr, 21:03, "Giampaolo Rodola'" <gne... at gmail.com> wrote:
> On 1 Apr, 18:27, "Steven Bethard" <steven.beth... at gmail.com> wrote:
>
> > On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
>
> > <facundobati... at gmail.com> wrote:
> > > 2008/4/1, Tim Golden <m... at timgolden.me.uk>:
> > > ?> ?If this is the thing to do, presumably test_support should
> > > ?> ?grow a "remove_file" which does something of this sort?
>
> > > ?+1 (I was thinking exactly that).
>
> > +1 here too. That looks like a great solution. ?Of course, once it's
> > in test_support, we need to fix *all* file removals in the test suite.
> > ;-)
>
> > Steve
>
> Why not just modifying test_support.unlink() like this?
> Surely more convenient than modifying the whole suite.
>
> def unlink(filename):
> ? ? try:
> ? ? ? ? if os.name == 'nt':
> ? ? ? ? ? ? os.rename(filename, filename + ".deleted")
> ? ? ? ? ? ? filename = filename + ".deleted"
> ? ? ? ? os.unlink(filename)
> ? ? except OSError:
> ? ? ? ? pass


Another solution, probably better:


def unlink(filename):
    try:
        os.unlink(filename)
    except OSError:
        if os.name == 'nt':
            try:
                os.rename(filename, filename + ".deleted")
                os.unlink(filename + ".deleted")
            except OSError:
                pass

From gnewsg at gmail.com  Tue Apr  1 21:26:17 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Tue, 1 Apr 2008 12:26:17 -0700 (PDT)
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F287A3.3090703@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com> 
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com> 
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk> 
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com> 
	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com> 
	<02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com> 
	<47F287A3.3090703@timgolden.me.uk>
Message-ID: <0a4b21eb-50cf-45fb-bc86-0e6a490bf84f@y24g2000hsd.googlegroups.com>



On 1 Apr, 21:06, Tim Golden <m... at timgolden.me.uk> wrote:
> Giampaolo Rodola' wrote:
>
> > On 1 Apr, 18:27, "Steven Bethard" <steven.beth... at gmail.com> wrote:
> >> On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
>
> >> <facundobati... at gmail.com> wrote:
> >>> 2008/4/1, Tim Golden <m... at timgolden.me.uk>:
> >>> ?> ?If this is the thing to do, presumably test_support should
> >>> ?> ?grow a "remove_file" which does something of this sort?
> >>> ?+1 (I was thinking exactly that).
> >> +1 here too. That looks like a great solution. ?Of course, once it's
> >> in test_support, we need to fix *all* file removals in the test suite.
> >> ;-)
>
> >> Steve
>
> > Why not just modifying test_support.unlink() like this?
> > Surely more convenient than modifying the whole suite.
>
> > def unlink(filename):
> > ? ? try:
> > ? ? ? ? if os.name == 'nt':
> > ? ? ? ? ? ? os.rename(filename, filename + ".deleted")
> > ? ? ? ? ? ? filename = filename + ".deleted"
> > ? ? ? ? os.unlink(filename)
> > ? ? except OSError:
> > ? ? ? ? pass
>
> Funnily enough, that's exactly what the patch I've
> put together does.

Sorry but maybe I misunderstood what you said above.
It seems to me you proposed to add a new "remove_file" function to
test_support while the solution I suggested was modifying the current
test_support.unlink() function in a similar manner you proposed and
have all tests use it wherever it is possible.

From mail at timgolden.me.uk  Tue Apr  1 21:47:29 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 01 Apr 2008 20:47:29 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <0a4b21eb-50cf-45fb-bc86-0e6a490bf84f@y24g2000hsd.googlegroups.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>
	<47F25F2E.4070408@timgolden.me.uk>
	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>
	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>
	<02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
	<47F287A3.3090703@timgolden.me.uk>
	<0a4b21eb-50cf-45fb-bc86-0e6a490bf84f@y24g2000hsd.googlegroups.com>
Message-ID: <47F29151.3000603@timgolden.me.uk>

Giampaolo Rodola' wrote:
>>> Why not just modifying test_support.unlink() like this?
>>> Surely more convenient than modifying the whole suite.
>>> def unlink(filename):
>>>     try:
>>>         if os.name == 'nt':
>>>             os.rename(filename, filename + ".deleted")
>>>             filename = filename + ".deleted"
>>>         os.unlink(filename)
>>>     except OSError:
>>>         pass

Tim Golden
>> Funnily enough, that's exactly what the patch I've
>> put together does.
> 
> Sorry but maybe I misunderstood what you said above.
> It seems to me you proposed to add a new "remove_file" function to
> test_support while the solution I suggested was modifying the current
> test_support.unlink() function in a similar manner you proposed and
> have all tests use it wherever it is possible.

I wasn't very clear. What I posted earlier was by way
of a proof-of-concept and did indeed use a "remove"
function. After other people's +1 I have put together
a possible patch which does what you're suggesting.
I haven't posted it yet and I need to work through
the test files checking what the impact would be.

I like your second suggestion BTW, altho' I'd
probably skip the "if os.name == 'nt'" line
and just give it a try in any case.

TJG

From mail at timgolden.me.uk  Tue Apr  1 21:58:53 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 01 Apr 2008 20:58:53 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F287A3.3090703@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<e04bdf310804010920m423bd12j272ccdba28099778@mail.gmail.com>	<d11dcfba0804010927k4fc4150fp4f0ad52c77a52aef@mail.gmail.com>	<02680572-1a12-4281-8abe-63b8a97b2ac9@i12g2000prf.googlegroups.com>
	<47F287A3.3090703@timgolden.me.uk>
Message-ID: <47F293FD.5010809@timgolden.me.uk>

Tim Golden wrote:
> Giampaolo Rodola' wrote:
>> On 1 Apr, 18:27, "Steven Bethard" <steven.beth... at gmail.com> wrote:
>>> On Tue, Apr 1, 2008 at 10:20 AM, Facundo Batista
>>>
>>> <facundobati... at gmail.com> wrote:
>>>> 2008/4/1, Tim Golden <m... at timgolden.me.uk>:
>>>>  >  If this is the thing to do, presumably test_support should
>>>>  >  grow a "remove_file" which does something of this sort?
>>>>  +1 (I was thinking exactly that).
>>> +1 here too. That looks like a great solution.  Of course, once it's
>>> in test_support, we need to fix *all* file removals in the test suite.
>>> ;-)
>>>
>>> Steve
>> Why not just modifying test_support.unlink() like this?
>> Surely more convenient than modifying the whole suite.
>>
>>
>> def unlink(filename):
>>     try:
>>         if os.name == 'nt':
>>             os.rename(filename, filename + ".deleted")
>>             filename = filename + ".deleted"
>>         os.unlink(filename)
>>     except OSError:
>>         pass
> 
> Funnily enough, that's exactly what the patch I've
> put together does. Have to look out, though for
> anywhere which just calls os.unlink rather than
> going through test_support.

Which turns out to be at least 117 instances:

C:\work-in-progress\python26\Lib\test>grep "os.unlink" *.py | wc -l
     117

(including the patch I just posted for os.access. Oops).
TJG


From schmir at gmail.com  Tue Apr  1 22:34:41 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 1 Apr 2008 22:34:41 +0200
Subject: [Python-Dev] test_signal on osx g4
In-Reply-To: <18418.20280.404970.167329@montanaro-dyndns-org.local>
References: <18418.20280.404970.167329@montanaro-dyndns-org.local>
Message-ID: <932f8baf0804011334n722a36ecgb581af9eb1e98447@mail.gmail.com>

this is http://bugs.python.org/issue1068268

On Tue, Apr 1, 2008 at 5:05 PM, <skip at pobox.com> wrote:

> test_signal is failing on osx g4:
>
>    test test_signal failed -- Traceback (most recent call last):
>      File
> "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 151,
> in test_main
>        self.fail(tb)
>    AssertionError: Traceback (most recent call last):
>      File
> "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 134,
> in test_main
>        self.run_test()
>      File
> "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 80,
> in run_test
>        child = subprocess.Popen(['kill', '-HUP', str(pid)])
>      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py",
> line 593, in __init__
>        errread, errwrite)
>      File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py",
> line 1078, in _execute_child
>        data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
>    OSError: [Errno 4] Interrupted system call
>
> This is the code which reads stderr from the child process:
>
>    data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
>    os.close(errpipe_read)
>    if data != "":
>        os.waitpid(self.pid, 0)
>        child_exception = pickle.loads(data)
>        raise child_exception
>
> I'm not an expert in this stuff my any stretch of the imagination, but
> shouldn't subprocess try harder to read this output?  Something like:
>
>    while True:
>        try:
>            data = os.read(errpipe_read, 1048576) # Exceptions limited to 1
> MB
>        except OSError, err:
>            if err.errno == errno.EINTR:
>                continue
>            else:
>                raise
>        else:
>            os.close(errpipe_read)
>            if data != "":
>                os.waitpid(self.pid, 0)
>                child_exception = pickle.loads(data)
>                raise child_exception
>            break
>
> Maybe not while True, but try a few times at least.
>

> Or is the system supposed to automatically restart interrupted system
> calls?
>

> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/schmir%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080401/75d5b1c0/attachment.htm 

From schmir at gmail.com  Tue Apr  1 22:45:56 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 1 Apr 2008 22:45:56 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
Message-ID: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>

Hi all,

anyone care to take a look at:
http://bugs.python.org/issue2014
It's about xmlrpclib not being able to send datetime objects with dates
before 1900.
I would like to see this go in and would also like to work on
http://bugs.python.org/issue1745722
(xmlrpc wsgi support).
But this only makes sense if someone also commits it...

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080401/47257f05/attachment.htm 

From amauryfa at gmail.com  Tue Apr  1 22:49:25 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 1 Apr 2008 22:49:25 +0200
Subject: [Python-Dev] Tracing at a more granular level (branch coverage)
In-Reply-To: <e36b84ee0804011202j374d0fa1j5fb8d654eb8b10e@mail.gmail.com>
References: <e36b84ee0804011202j374d0fa1j5fb8d654eb8b10e@mail.gmail.com>
Message-ID: <e27efe130804011349w729364a1x7d6bc00ee8af8296@mail.gmail.com>

Hello,

On Tue, Apr 1, 2008 at 9:02 PM, m h wrote:
> Howdy Folks-
>
>  I hope this is not too off topic.  Sadly the current sys.settrace only
>  allows tracing at the line level.  This isn't sufficient to do branch
>  and path coverage.  The main problem is that many boolean operations
>  can appear on a single line, and so you can't be sure which
>  conditional branch was taken using the current tracing method.
>
>  After speaking about code complexity/testing at Pycon [0], with a few
>  people there, and seeing a few people have made some strides or shown
>  interested in metrics/code graphs/flow and branch coverage I think
>  that there is interest in this.
>
>  One proposed method of getting branch coverage was to use Dalke's
>  Python4Ply [1] to translate code so that all branches occur on their
>  own line.  Then using line coverage on that and converting it back to
>  branch coverage of the original code.
>
>  The maintainer of coverage.py suggested that we look into patching
>  python instead to trace at a more granular level.  His feeling was
>  that there would be too many corner cases and the translation would
>  get hairy quite quickly.
>
>  Sadly in my 8 years of python experience I've yet to touch any c based
>  guts of python.  I'm looking for advice on how to get finer grain
>  tracing with sys.settrace.
>
>  Any advice or suggestions?  There is a quorum of people (at least 5
>  others) who would be very interested in this functionality because it
>  could lead to some cool tools built on top of it.  (With the end goal
>  that python code be cleaner, simpler and better tested).

You can start by looking at the file Python/ceval.c, which contains
the interpreter loop.
More precisely, the function maybe_call_line_trace() is responsible to call
sys.settrace. I think you will have to change the logic there.

-- 
Amaury Forgeot d'Arc

From martin at v.loewis.de  Tue Apr  1 23:48:35 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 01 Apr 2008 23:48:35 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
Message-ID: <47F2ADB3.8050802@v.loewis.de>

> anyone care to take a look at:
> http://bugs.python.org/issue2014
> It's about xmlrpclib not being able to send datetime objects with dates
> before 1900.
> I would like to see this go in and would also like to work on
> http://bugs.python.org/issue1745722
> (xmlrpc wsgi support).
> But this only makes sense if someone also commits it...

Can you please explain why this is an important problem?
Dates before 1900 have all passed long ago, so they shouldn't
occur that often in real applications.

Regards,
Martin

From martin at v.loewis.de  Wed Apr  2 01:12:35 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2008 01:12:35 +0200
Subject: [Python-Dev] FW: [issue2513] 64bit cross compilation on windows
In-Reply-To: <00c801c893dd$32d03320$98709960$@com.au>
References: <002401c892ff$a8254630$f86fd290$@com.au>
	<00c801c893dd$32d03320$98709960$@com.au>
Message-ID: <47F2C163.9000402@v.loewis.de>

> Currently, the "official" (by way of being de-facto) directory structure for
> a build tree is 'PCBuild/.' for x86 builds and 'PCBuild/amd64' for x64
> platforms.  I believe this might cause problems for people trying to port
> their applications to 64bit platforms.  My proposal is that we change this
> subtly - that both 'PCBuild/x86', and 'PCBuild/amd64' exist, while
> 'PCBuild/.' is always the 'native' platform - ie, that 'PCBuild/.' be a copy
> of one of the platform subdirectories.

Can you explain what you mean by "native platform"? I usually perform
cross-builds for amd64, so would then this directory remain empty?

> The rationale is fairly simple: I'm quite certain that this new directory
> layout will break existing "native only" build processes (ie, those that
> aren't aware of cross-compilation).  I'm quite certain that Mozilla will
> break, for example, and no cross-compilation process exists that I am aware
> of.  Existing build tools already know about the PCBuild directory, and are
> focused almost exclusively towards native compilation - all such tools on
> non-x86 platforms are currently broken.

The reverse may also be true: some tools may expect PCbuild to contain
always x86 binaries, even on AMD64 - as they don't support non-x86 at
all. Those tools might break if PCbuild suddenly starts containing
AMD64 binaries.

If this is all about *just* Mozilla, then I'd be much more in favour
than if it was for some theoretical package. So if Mozilla builds with
that kind of setup, and works correctly - go for it. If you have to
fix Mozilla anyway, consider fixing it so it looks in the amd64
directory.

Regards,
Martin


From guido at python.org  Wed Apr  2 02:09:24 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 1 Apr 2008 17:09:24 -0700
Subject: [Python-Dev] Thread-safe file objects, the return
In-Reply-To: <loom.20080331T194117-885@post.gmane.org>
References: <loom.20080331T194117-885@post.gmane.org>
Message-ID: <ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>

This is not something that keeps me awake at night, but I am aware of
it. Your solution (a counter) seems fine except I think perhaps the
close() call should not raise IOError -- instead, it should set a flag
so that the thread that makes the counter go to zero can close the
thread (after all the file got closed while it was being used).

There are of course other concurrency issues besides close -- what if
two threads both try to do I/O on the file? What will the C stdio
library do in that case? Are stdio files thread-safe at the C level?
So (classically contradicting myself while I think the problem over
more) perhaps any I/O operation should be disallowed while the file is
in use by another thread?

--Guido

On Mon, Mar 31, 2008 at 1:09 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>  Hello,
>
>  It seems this subject has had quite a bit of history. Tim Peters demonstrated
>  the problem in 2003 in this message:
>  http://mail.python.org/pipermail/python-dev/2003-June/036537.html
>
>  In short, Python file objects release the GIL before calling any C stdlib
>  function on their embedded FILE pointer. Unfortunately, if another thread
>  calls fclose on the FILE pointer concurrently, the contents pointed to can
>  become garbage and the interpreter process crashes. Just by using the same
>  file object in two threads running pure Python code, you can crash the
>  interpreter.
>
>  (another, easier-to-solve problem is that the FILE pointer stored in the
>  file object could become NULL at the point it is used by another thread.
>  If that was the only problem you could just store the FILE pointer in a
>  local variable before releasing the GIL et voil?)
>
>  There was some discussion at the time about the possible resolution. I've
>  tried to fix the problem, and I've come to what I think is a satisfying
>  solution, which I can sum up as the following bullet points:
>   * Each file object gets a dedicated counter, which is incremented before
>  the bject releases the GIL and decremented after the GIL is taken again; thus
>  this counter keeps track of how many running "unlocked" sections of code are
>  using that particular file object. (please note the counter doesn't need its
>  own lock, since it is only modified in GIL-protected sections)
>   * In the close() method, if the aforementioned counter is greater than 0,
>  we refuse to call fclose and instead raise an IOError.
>
>  This may seem like a worrying semantic change, but I don't think it is, for the
>  following reasons:
>   1) if we closed the FILE pointer anyway, the interpreter would likely crash
>  because another thread would be using garbage data (that's what we are trying
>  to fix after all!)
>   2) if close() raises an IOError, it can be called again later, or at worse
>  fclose will be called when the file object is garbage collected
>   3) close() can already raise an IOError if fclose fails for whatever reason
>  (although for sure it's probably very rare)
>   4) it doesn't seem wrong to notify the programmer that his code is very
>  unsafe
>
>  The patch is attached at http://bugs.python.org/issue815646 . It addresses
>  (or at least I hope it does) all potential problems with pure Python code,
>  threads, and the file object. It doesn't try to fix C extensions using the
>  PyFile_AsFile API and doing their own dirty things with the FILE pointer. It
>  could be a second step if the approach is accepted, but as noted in the 2003
>  discussions it would probably involve a new API. Whether we want to introduce
>  such an API in Python 2.x while Python 3.0 has a different IO model anyway
>  is left open to discussion :)
>
>  Regards
>
>  Antoine.
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mhammond at skippinet.com.au  Wed Apr  2 03:26:22 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Wed, 2 Apr 2008 12:26:22 +1100
Subject: [Python-Dev] [Distutils] FW: [issue2513] 64bit
	cross	compilation on windows
In-Reply-To: <47F2C163.9000402@v.loewis.de>
References: <002401c892ff$a8254630$f86fd290$@com.au>	<00c801c893dd$32d03320$98709960$@com.au>
	<47F2C163.9000402@v.loewis.de>
Message-ID: <016001c89460$944f10c0$bced3240$@com.au>

Martin quoting me:

> > Currently, the "official" (by way of being de-facto) directory
> > structure for a build tree is 'PCBuild/.' for x86 builds and 
> > 'PCBuild/amd64' for x64
> > platforms.  I believe this might cause problems for people trying to
> > port their applications to 64bit platforms.  My proposal is that we 
> > change this subtly - that both 'PCBuild/x86', and 'PCBuild/amd64' 
> > exist, while 'PCBuild/.' is always the 'native' platform - ie, 
> > that 'PCBuild/.' be a copy of one of the platform subdirectories.
> 
> Can you explain what you mean by "native platform"? I usually perform
> cross-builds for amd64, so would then this directory remain empty?

By "native platform", I mean it contains binaries that are native for the
current platform.  IOW, on x86 platforms, PCBuild would contain x86
binaries, and binaries compiled for AMD64 would live in the AMD64 directory.
 
If in a given directory tree, you *only* did a cross-compile, then yes,
PCBuild would remain empty - but I don't think that is a problem; tools that
know how to cross-compile (such as the distutils after my patch is applied)
will not look in the PCBuild directory, and tools that do not cross-compile
will be unable to use the x64 binaries from the x86 platform anyway.  In
other words, I can't think of anything that would break by having an empty
PCBuild directory in a tree that only contains binaries that can't be
executed on the current platform.

> > The rationale is fairly simple: I'm quite certain that this new
> > directory layout will break existing "native only" build processes 
> > (ie, those that aren't aware of cross-compilation).  I'm quite 
> > certain that Mozilla will break, for example, and no 
> > cross-compilation process exists that I am aware
> > of.  Existing build tools already know about the PCBuild directory,
> > and are focused almost exclusively towards native compilation - all 
> > such tools on non-x86 platforms are currently broken.
> 
> The reverse may also be true: some tools may expect PCbuild to contain
> always x86 binaries, even on AMD64 - as they don't support non-x86 at
> all. Those tools might break if PCbuild suddenly starts containing
> AMD64 binaries.

I agree.  However, it is my assertion that there are very few build tools
which expect the layout you describe, simply as the scheme has only been
around for a few months, and only in Python 2.6 builds.  Further, I assert
that there are a greater number of build tools which do not support
cross-compilation, but will build natively on x64 and expect 'PCBuild' to
have libraries they can link with to create an x64 binary.

I accept my assertions may be incorrect - in which case keeping the existing
layout is indeed the best options.  But I assert that my assertions are
correct :)

> If this is all about *just* Mozilla, then I'd be much more in favour
> than if it was for some theoretical package. So if Mozilla builds with
> that kind of setup, and works correctly - go for it. If you have to
> fix Mozilla anyway, consider fixing it so it looks in the amd64
> directory.

Nope - it's not about *just* Mozilla at all, and I don't expect that I
personally will be involved in any 64bit issues for that platform.  That
just happens to be one build process that I know about, but I except it does
some "typical" things other tools might do - like executing the python
executable to sniff sys.prefix and looking for (eg) "Libs" and "PCBuild" to
locate libraries to link with seems fairly likely to me.  Even looking for
the executable itself directly in PCBuild seems likely.  On an x64 platform,
such tools may well find the 32bit version, and it may even seem to work -
but it is unlikely to be doing what they expect.  Or they may find no binary
at all.

It seems that you previously agreed with this.  From
http://mail.python.org/pipermail/distutils-sig/2007-May/007668.html; you are
replying to (I think) Kristj?n:

| > I am baffled about why the build environment's layout matters,
| > but once an .msi install can place the binaries in any
| > old place it wants.  The build structure doesn't have to
| > reflect the final installed structure at all.

| No. But still, people like to be able to "run" Python out of
| a source check-out. This has been supported for a long time,
| and more and more stuff was added to support it. For examples
| within Python itself, see the support in distutils, getpathp.c,
| PCbuild/rt.bat, and Tools/buildbot/*.bat. Reportedly (by
| Mark), building Mozilla (the web browser) also "knows"
| about PCbuild.

Your comments exactly reflect my concern, and I believe them relevant for
people working with native binaries on x64.  But as implied above, I
actually have zero personal interest in this at the moment - unlike the
distutils cross-compile patch, which does scratch an itch of mine.  It just
seems it would make life easier for people down the track - but I'm happy to
let it go if your position has changed or I have mis-understood it.

Thanks,

Mark



From martin at v.loewis.de  Wed Apr  2 04:15:22 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2008 04:15:22 +0200
Subject: [Python-Dev] [Distutils] FW: [issue2513] 64bit
 cross	compilation on windows
In-Reply-To: <016001c89460$944f10c0$bced3240$@com.au>
References: <002401c892ff$a8254630$f86fd290$@com.au>	<00c801c893dd$32d03320$98709960$@com.au>
	<47F2C163.9000402@v.loewis.de>
	<016001c89460$944f10c0$bced3240$@com.au>
Message-ID: <47F2EC3A.9090300@v.loewis.de>

>> The reverse may also be true: some tools may expect PCbuild to contain
>> always x86 binaries, even on AMD64 - as they don't support non-x86 at
>> all. Those tools might break if PCbuild suddenly starts containing
>> AMD64 binaries.
> 
> I agree.  However, it is my assertion that there are very few build tools
> which expect the layout you describe, simply as the scheme has only been
> around for a few months, and only in Python 2.6 builds.  

That PCbuild exists and contains x86 binaries? This scheme has been in
place ever since the PCbuild directory got created, several years ago!

> Nope - it's not about *just* Mozilla at all, and I don't expect that I
> personally will be involved in any 64bit issues for that platform.  That
> just happens to be one build process that I know about, but I except it does
> some "typical" things other tools might do - like executing the python
> executable to sniff sys.prefix and looking for (eg) "Libs" and "PCBuild" to
> locate libraries to link with seems fairly likely to me.  Even looking for
> the executable itself directly in PCBuild seems likely.  On an x64 platform,
> such tools may well find the 32bit version, and it may even seem to work -
> but it is unlikely to be doing what they expect.

Unless the *want* the x86 binaries, of course, which always had been in
this place.

> Your comments exactly reflect my concern, and I believe them relevant for
> people working with native binaries on x64.

No, I never cared about what binaries are "native". Instead, I wanted to
see those binaries in PCbuild which had been built last. If you just
built the x86 binaries, PCbuild should contain x86 binaries, even if
you were running Win64.

This is different from what you propose, but only slightly so (but
perhaps importantly, depending on the scenario).

> But as implied above, I
> actually have zero personal interest in this at the moment - unlike the
> distutils cross-compile patch, which does scratch an itch of mine.  It just
> seems it would make life easier for people down the track - but I'm happy to
> let it go if your position has changed or I have mis-understood it.

OK, if we don't have an actual use case, I suggest to leave things as
is. People running into this problem should propose solutions themselves.

Regards,
Martin

From jyasskin at gmail.com  Wed Apr  2 06:10:22 2008
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Tue, 1 Apr 2008 21:10:22 -0700
Subject: [Python-Dev] test_signal on osx g4
In-Reply-To: <18418.20280.404970.167329@montanaro-dyndns-org.local>
References: <18418.20280.404970.167329@montanaro-dyndns-org.local>
Message-ID: <5d44f72f0804012110s32b43d5erc14a7b3afeb08f3b@mail.gmail.com>

I've tried to fix test_signal at least. For those particular calls,
EINTR means that the kill I launched finished before I was expecting,
so we can ignore it without retrying. Figuring out subprocess in
general is a worthy goal but shouldn't block making the test less
flaky. :) r62102.

On Tue, Apr 1, 2008 at 8:05 AM,  <skip at pobox.com> wrote:
> test_signal is failing on osx g4:
>
>     test test_signal failed -- Traceback (most recent call last):
>       File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 151, in test_main
>         self.fail(tb)
>     AssertionError: Traceback (most recent call last):
>       File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 134, in test_main
>         self.run_test()
>       File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/test/test_signal.py", line 80, in run_test
>         child = subprocess.Popen(['kill', '-HUP', str(pid)])
>       File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py", line 593, in __init__
>         errread, errwrite)
>       File "/Users/buildslave/bb/trunk.psf-g4/build/Lib/subprocess.py", line 1078, in _execute_child
>         data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
>     OSError: [Errno 4] Interrupted system call
>
>  This is the code which reads stderr from the child process:
>
>     data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
>     os.close(errpipe_read)
>     if data != "":
>         os.waitpid(self.pid, 0)
>         child_exception = pickle.loads(data)
>         raise child_exception
>
>  I'm not an expert in this stuff my any stretch of the imagination, but
>  shouldn't subprocess try harder to read this output?  Something like:
>
>     while True:
>         try:
>             data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
>         except OSError, err:
>             if err.errno == errno.EINTR:
>                 continue
>             else:
>                 raise
>         else:
>             os.close(errpipe_read)
>             if data != "":
>                 os.waitpid(self.pid, 0)
>                 child_exception = pickle.loads(data)
>                 raise child_exception
>             break
>
>  Maybe not while True, but try a few times at least.
>
>  Or is the system supposed to automatically restart interrupted system calls?
>
>  Skip
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com
>



-- 
Namast?,
Jeffrey Yasskin
http://jeffrey.yasskin.info/

From greg at krypto.org  Wed Apr  2 08:19:17 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 1 Apr 2008 23:19:17 -0700
Subject: [Python-Dev] Thread-safe file objects, the return
In-Reply-To: <ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>
References: <loom.20080331T194117-885@post.gmane.org>
	<ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>
Message-ID: <52dc1c820804012319q6b548fb1r28df92ab7dcc319e@mail.gmail.com>

On Tue, Apr 1, 2008 at 5:09 PM, Guido van Rossum <guido at python.org> wrote:

> This is not something that keeps me awake at night, but I am aware of
> it. Your solution (a counter) seems fine except I think perhaps the
> close() call should not raise IOError -- instead, it should set a flag
> so that the thread that makes the counter go to zero can close the
> thread (after all the file got closed while it was being used).
>

No, raising IOError is the right thing to do here.

The problem is that calling close on a file implies that the close actually
completed in the OS when it returns.  The call should not return until the
file object has actually been closed in the underlying layers.  You can't
leave it for later to be done by one of the other currently operating
threads as you violate the close semantics and lose direct indication of an
error that occurred during close.


> There are of course other concurrency issues besides close -- what if
> two threads both try to do I/O on the file? What will the C stdio
> library do in that case? Are stdio files thread-safe at the C level?
> So (classically contradicting myself while I think the problem over
> more) perhaps any I/O operation should be disallowed while the file is
> in use by another thread?
>

that does sound like the safest thing to do...

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080401/908e21e2/attachment-0001.htm 

From greg at krypto.org  Wed Apr  2 08:26:20 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 1 Apr 2008 23:26:20 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F25F2E.4070408@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
Message-ID: <52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>

On Tue, Apr 1, 2008 at 9:13 AM, Tim Golden <mail at timgolden.me.uk> wrote:

> Tim Golden wrote:
> > Steven Bethard wrote:
> >> At the sprints, I ran into a bunch of similar errors running the test
> >> suite on my Windows Vista box, even on tests that were properly
> >> cleaning up after themselves in tearDown(). I even tried putting in
> >> sleeps as long as 1 second, to no avail. The only way to get the test
> >> suite to run without these errors was to stop the Windows Search
> >> Service, fully disable Icon Overlays for TortoiseSVN, and then close
> >> down all open folders.
> >>
> >> Any chance the boxes the tests are being run on are running the
> >> Windows Search Service or have Icon Overlays enabled for TortoiseSVN?
> >>
> >> (If anyone has any ideas of how to get around these problems, I'd love
> >> to hear them. Seems like I shouldn't have to disable these services.)
> >
> > I'm not sure I'm going to help here, but the reason that
> > this happens is that certain services -- and the Desktop
> > Search tools are notorious for this -- get file handles with
> > FILE_SHARE_DELETE, which means that the file isn't actually
> > removed when it's deleted; only when the last of those handles
> > closes.
> >
> > I did see a suggestion somewhere that, to circumvent this issue,
> > instead of removing it a file you need to move it to %TEMP%
> > (or wherever) and delete it there. I'll try to knock up a test
> > case to see if this works.
>
> OK: to confirm, the following test seems to indicate that
> doing a (local - same volume) rename followed by a remove
> will do the right thing, even when a FILE_SHARE_DELETE handle
> is held.
>
> If this is the thing to do, presumably test_support should
> grow a "remove_file" which does something of this sort?
>
> TJG
>
> <code>
> import os, sys
> import win32file
>
> FILENAME = "test"
>
> def rename_and_remove (filename):
>   os.rename (filename, filename + ".deleted")
>   os.remove (filename + ".deleted")


Isn't this still going to run into problems when the rename fails because
the earlier tests remove still left the .deleted file around due to some
other running desktop search service that now has the .deleted file open?

+1 on not supporting OSes with such bad filesystem semantics? ;)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080401/c87dfe84/attachment.htm 

From tnelson at onresolve.com  Wed Apr  2 09:00:41 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 00:00:41 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B1@EXMBX04.exchhosting.com>


> def rename_and_remove (filename):
>  os.rename (filename, filename + ".deleted")
>  os.remove (filename + ".deleted")

> Isn't this still going to run into problems when the rename
> fails because the earlier tests remove still left the .deleted
> file around due to some other running desktop search service
> that now has the .deleted file open?

I haven't looked into all the various places the tests write temp files to, but if we could localise everything to a common root directory, i.e. %TEMP%\python-regrtest, we could then attempt to blow this away at the start of regrtest.py before any tests run, and refuse to run if this fails.  This would be in combination with the unlinking/renaming approach discussed.  This approach seems like it would cover all bases a bit more effectively.

        Trent.

From tnelson at onresolve.com  Wed Apr  2 09:12:05 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 00:12:05 -0700
Subject: [Python-Dev] No time for svn merge
In-Reply-To: <47F07068.3060503@v.loewis.de>
References: <fsojc4$dm9$1@ger.gmane.org>	<47EFFD64.8020006@v.loewis.de>
	<1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com>
	<47F00C33.1050503@v.loewis.de>
	<5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com>
	<47F0113D.8020408@v.loewis.de>
	<5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com>
	<47F07068.3060503@v.loewis.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B6@EXMBX04.exchhosting.com>

> > Yes, that's all I meant:  make it the committer's job
> > to merge or block as appropriate.  I just wasn't sure if
> > there was some reason that this would be difficult or
> > undesirable.
>
> Ah, yes. It is indeed difficult or undesirable, or was
> so in the past: Some committers don't care (didn't care)
> at all about 3k. They would have to setup sandboxes,
> learn what the nature of changes is, and invest some
> regular time into forward-porting.

Is this *really* the case still?  Who are these rogue committers? ;-)

I think holding a developer accountable for merging or blocking to py3k when they commit to trunk is a great idea.  Who better to pass judgement on such an activity than the person closest to it?

        Trent.


From tnelson at onresolve.com  Wed Apr  2 09:37:14 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 00:37:14 -0700
Subject: [Python-Dev] [Distutils] FW: [issue2513]
	64bit	cross	compilation on windows
In-Reply-To: <016001c89460$944f10c0$bced3240$@com.au>
References: <002401c892ff$a8254630$f86fd290$@com.au>
	<00c801c893dd$32d03320$98709960$@com.au>	<47F2C163.9000402@v.loewis.de>
	<016001c89460$944f10c0$bced3240$@com.au>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BA@EXMBX04.exchhosting.com>

> Further, I
> assert that there are a greater number of build tools which do not support
> cross-compilation, but will build natively on x64 and expect 'PCBuild'
> to have libraries they can link with to create an x64 binary.

I'm with Martin on this one as well I think.  If I understand correctly, you're proposing:

PCbuild                 - may have contents of x86 or x64 depending
                          on the build machine's architecture
PCbuild/amd64   - always x64
PCbuild/x86             - always x86

And what we've currently got is:

PCbuild/                - always x86
PCbuild/amd64   - always x64

I think this is fine; we don't really have a notion of compiling for a native platform, nor is the build machine's architecture factored into the equation.  I believe this is a Good Thing.  If you want a 32-bit build, use the 'Win32' target platform in VS; if you want a 64-bit build, use 'x64'.

        Trent.



From tnelson at onresolve.com  Wed Apr  2 09:58:35 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 00:58:35 -0700
Subject: [Python-Dev] [Python-3000] the release gods are angry at python
In-Reply-To: <fsdu7o$fo1$1@ger.gmane.org>
References: <ee2a432c0803252321g17ca6027ma7d2391d015e8bce@mail.gmail.com>
	<fsdq6u$vp6$1@ger.gmane.org>	<fsdqpk$2h4$1@ger.gmane.org>
	<47EA72D4.8000709@cheimes.de> <fsdu7o$fo1$1@ger.gmane.org>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BF@EXMBX04.exchhosting.com>

> > In the py3k branch I've assigned the audio resource to the winsound
> > tests. Only regrtest.py -uall or -uaudio runs the winsound test.
> Reason:
> > the test sound was freaking out my poor cat. :/
>
> I feel with your cat ;-).
> This would not help on the buildbot since it runs 'rt.bat -d -q -uall -
> rw'.

I feel for the poor NOC engineers at my colo that freak out when some random server in a farm of thousands starts making bizarre sounds.

I detest test_winsound.  There are so many corner cases you need to account for that makes the test pointless as you end up wrapping everything in except: pass blocks.  Does the system have a legacy beep driver?  Is it enabled?  Is it disabled?  Is there a sound card?  Is it enabled or disabled?  Pah!

+1 to removing audio out of -uall, if only for the sake of cats, erroneously red buildbots, and poor ServerCentral NOC engineers.

        Trent.


From mail at timgolden.me.uk  Wed Apr  2 10:03:51 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 02 Apr 2008 09:03:51 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>
	<47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
Message-ID: <47F33DE7.3030702@timgolden.me.uk>

Gregory P. Smith wrote:
> On Tue, Apr 1, 2008 at 9:13 AM, Tim Golden <mail at timgolden.me.uk> wrote:
> 
>> Tim Golden wrote:

>> <code>
>> import os, sys
>> import win32file
>>
>> FILENAME = "test"
>>
>> def rename_and_remove (filename):
>>   os.rename (filename, filename + ".deleted")
>>   os.remove (filename + ".deleted")

> Isn't this still going to run into problems when the rename fails because
> the earlier tests remove still left the .deleted file around due to some
> other running desktop search service that now has the .deleted file open?

I admit: this did occur to me on the train this am. While I
try to think of a robust way to handle this, other people have
proposed variations on pid-based / tempdir based filenames instead
of the same name for each test. In principle this sounds good to me,
but I'm not at all well-placed to assess the impact it might have
on the unit tests in general.

TJG

From lists at cheimes.de  Wed Apr  2 10:57:41 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 02 Apr 2008 10:57:41 +0200
Subject: [Python-Dev] No time for svn merge
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B6@EXMBX04.exchhosting.com>
References: <fsojc4$dm9$1@ger.gmane.org>	<47EFFD64.8020006@v.loewis.de>	<1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com>	<47F00C33.1050503@v.loewis.de>	<5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com>	<47F0113D.8020408@v.loewis.de>	<5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com>
	<47F07068.3060503@v.loewis.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B6@EXMBX04.exchhosting.com>
Message-ID: <47F34A85.5020909@cheimes.de>

Trent Nelson schrieb:
> I think holding a developer accountable for merging or blocking to py3k when they commit to trunk is a great idea.  Who better to pass judgement on such an activity than the person closest to it?

Blocking a revision makes my job as The Merger easier.

I'm not so sure about the merging part. It takes some experience with
the Python 3.0 code base to know the subtle differences in the C API.
Most merges are straight forward for me. If you enforce the forward
merging on every developer it may slow down development.
Each regular merge takes me about 45 minutes of computer time but less
than 15 supervisor time. The computer time is mostly compile and test
time in the background. If everybody merges her own code to 3.0 it still
takes *everybody* about 10 minutes of time and 45 minutes of computer time.

Christian

From amauryfa at gmail.com  Wed Apr  2 11:07:03 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 2 Apr 2008 11:07:03 +0200
Subject: [Python-Dev] [Python-3000] the release gods are angry at python
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BF@EXMBX04.exchhosting.com>
References: <ee2a432c0803252321g17ca6027ma7d2391d015e8bce@mail.gmail.com>
	<fsdq6u$vp6$1@ger.gmane.org> <fsdqpk$2h4$1@ger.gmane.org>
	<47EA72D4.8000709@cheimes.de> <fsdu7o$fo1$1@ger.gmane.org>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BF@EXMBX04.exchhosting.com>
Message-ID: <e27efe130804020207v6d4c3e30y49197077d19fbc8b@mail.gmail.com>

On Wed, Apr 2, 2008 at 9:58 AM, Trent Nelson <tnelson at onresolve.com> wrote:
> > > In the py3k branch I've assigned the audio resource to the winsound
>  > > tests. Only regrtest.py -uall or -uaudio runs the winsound test.
>  > Reason:
>  > > the test sound was freaking out my poor cat. :/
>  >
>  > I feel with your cat ;-).
>  > This would not help on the buildbot since it runs 'rt.bat -d -q -uall -
>  > rw'.
>
>  I feel for the poor NOC engineers at my colo that freak out when some random server in a farm of thousands starts making bizarre sounds.
>
>  I detest test_winsound.  There are so many corner cases you need to account for that makes the test pointless as you end up wrapping everything in except: pass blocks.  Does the system have a legacy beep driver?  Is it enabled?  Is it disabled?  Is there a sound card?  Is it enabled or disabled?  Pah!
>
>  +1 to removing audio out of -uall, if only for the sake of cats, erroneously red buildbots, and poor ServerCentral NOC engineers.

And I would not mind removing this module altogether, and provide a
ctypes implementation.

-- 
Amaury Forgeot d'Arc

From ncoghlan at gmail.com  Wed Apr  2 11:13:24 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 02 Apr 2008 19:13:24 +1000
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F33DE7.3030702@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk>
Message-ID: <47F34E34.9010507@gmail.com>

Tim Golden wrote:
> I admit: this did occur to me on the train this am. While I
> try to think of a robust way to handle this, other people have
> proposed variations on pid-based / tempdir based filenames instead
> of the same name for each test. In principle this sounds good to me,
> but I'm not at all well-placed to assess the impact it might have
> on the unit tests in general.

Personally, I've never really understood the purpose of 
test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
Tests using that module don't care if the old files take 'a while' to 
get deleted on Windows, as tempfile uses a different name each time anyway.

Is using a fixed TESTFN just an old approach that predates the existence 
of a robust tempfile module in the standard library?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mail at timgolden.me.uk  Wed Apr  2 11:24:22 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 02 Apr 2008 10:24:22 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F34E34.9010507@gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>
	<47F34E34.9010507@gmail.com>
Message-ID: <47F350C6.3010209@timgolden.me.uk>

Nick Coghlan wrote:
> Tim Golden wrote:
>> I admit: this did occur to me on the train this am. While I
>> try to think of a robust way to handle this, other people have
>> proposed variations on pid-based / tempdir based filenames instead
>> of the same name for each test. In principle this sounds good to me,
>> but I'm not at all well-placed to assess the impact it might have
>> on the unit tests in general.
> 
> Personally, I've never really understood the purpose of 
> test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
> just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
> Tests using that module don't care if the old files take 'a while' to 
> get deleted on Windows, as tempfile uses a different name each time anyway.
> 
> Is using a fixed TESTFN just an old approach that predates the existence 
> of a robust tempfile module in the standard library?

I'm a neophyte when it comes to core development, so I've simply
cloned existing tests, assumed that there was some kind of (possibly 
unwritten) standard which used test_support.TESTFN. As I look at
it, though it seems a slightly odd choice, although it has variants
for testing unicode filenames specifically which I imagine are
useful in some places.

I'm perfectly happy to run through the test suite, patching it one
way or another. The trouble is that I've little confidence that I
can assess whether or not such a change will have affected the
actual meaning of a test. And, since these are tests,
Quis custodiet...?

TJG

From lists at cheimes.de  Wed Apr  2 11:42:04 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 02 Apr 2008 11:42:04 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <47F2ADB3.8050802@v.loewis.de>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<47F2ADB3.8050802@v.loewis.de>
Message-ID: <fsvk3p$mge$1@ger.gmane.org>

Martin v. L?wis schrieb:
> Can you please explain why this is an important problem?
> Dates before 1900 have all passed long ago, so they shouldn't
> occur that often in real applications.

Does xmlrpc support dates for 1900? For historic dates the Julian Day
Number family (MJD or JDN) or Rata Die are more appropriate and much
easier to use. I wish somebody could add both to the datetime module.

Christian


From mail at timgolden.me.uk  Wed Apr  2 12:00:13 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 02 Apr 2008 11:00:13 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F33DE7.3030702@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk>
Message-ID: <47F3592D.3080700@timgolden.me.uk>

Tim Golden wrote:
> Gregory P. Smith wrote:
>> On Tue, Apr 1, 2008 at 9:13 AM, Tim Golden <mail at timgolden.me.uk> wrote:
>>
>>> Tim Golden wrote:
> 
>>> <code>
>>> import os, sys
>>> import win32file
>>>
>>> FILENAME = "test"
>>>
>>> def rename_and_remove (filename):
>>>   os.rename (filename, filename + ".deleted")
>>>   os.remove (filename + ".deleted")
> 
>> Isn't this still going to run into problems when the rename fails because
>> the earlier tests remove still left the .deleted file around due to some
>> other running desktop search service that now has the .deleted file open?

I've looked around this issue as far as I can, and I can't
come up with a foolproof way of guaranteeing that this won't
happen. I considered, eg, trying to open the test file with
semantics which preclude FILE_SHARE_DELETE from succeeding
(ie stopping the search services / TSVN from getting a handle
at all) but I can't find anything which doesn't cause more
problems than it solves.

Lest this discussion descend into inactivity, I've done a
fresh checkout and I'm producing a patch which will add
open_testfile and close_testfile functions to test_support
(or something like that) and which will update all the tests
to use those. I'm inclined to go with the tempfile approach
but at least having the functions means that that could
change.

This is obviously a wide-reaching change, but unless anyone's
got better ideas, the alternative seems to be intermittently-failing
test runs on the Windows buildbots.

TJG

From solipsis at pitrou.net  Wed Apr  2 12:17:48 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 2 Apr 2008 10:17:48 +0000 (UTC)
Subject: [Python-Dev] Thread-safe file objects, the return
References: <loom.20080331T194117-885@post.gmane.org>
	<ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>
Message-ID: <loom.20080402T100849-143@post.gmane.org>


Guido van Rossum <guido <at> python.org> writes:
> Your solution (a counter) seems fine except I think perhaps the
> close() call should not raise IOError -- instead, it should set a flag
> so that the thread that makes the counter go to zero can close the
> thread (after all the file got closed while it was being used).

I agree with Gregory: we should be explicit about what happens. I wonder
what we would gain from that approach - apart from encouraging dangerous
coding practices :)
It also depends how far we want to go: I am merely proposing to fix the 
crashes, do we want to provide a "smarter" close() variation that does what
you suggest for people that want (or need) to take the risk?

> There are of course other concurrency issues besides close -- what if
> two threads both try to do I/O on the file? What will the C stdio
> library do in that case? Are stdio files thread-safe at the C level?

According to the glibc documentation, at
http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html :

? The POSIX standard requires that by default the stream operations are
atomic. I.e., issuing two stream operations for the same stream in two
threads at the same time will cause the operations to be executed as if
they were issued sequentially. The buffer operations performed while
reading or writing are protected from other uses of the same stream. To do
this each stream has an internal lock object which has to be (implicitly)
acquired before any work can be done. ?

So according to POSIX rules it should be perfectly safe.
In any case, someone would have to try my patch under Windows and OS X and 
see if test_file.py passes without crashing.

Regards

Antoine.



From schmir at gmail.com  Wed Apr  2 12:45:31 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Wed, 2 Apr 2008 12:45:31 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <fsvk3p$mge$1@ger.gmane.org>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<47F2ADB3.8050802@v.loewis.de> <fsvk3p$mge$1@ger.gmane.org>
Message-ID: <932f8baf0804020345k7fdbb0c6k9f930fd88631f134@mail.gmail.com>

On Wed, Apr 2, 2008 at 11:42 AM, Christian Heimes <lists at cheimes.de> wrote:

> Martin v. L?wis schrieb:
> > Can you please explain why this is an important problem?
> > Dates before 1900 have all passed long ago, so they shouldn't
> > occur that often in real applications.
>

In the application where I needed it, the customer wanted to send/store
dates for e.g. the date of birth of some people.


>
> Does xmlrpc support dates for 1900? For historic dates the Julian Day


The xmlrpc spec says dates should be sent in the following format:

 <dateTime.iso8601>19980717T14:08:55</dateTime.iso8601>
1900 is a rather arbitrary limit with this format.

Note that the unpatched xmlrpclib is able to receive datetime objects with
dates before 1900:
~/ /usr/bin/python2.5                                               ralf at redok
Python 2.5.2 (r252:60911, Mar  9 2008, 11:14:55)
[GCC 4.2.3 (Debian 4.2.3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xmlrpclib, datetime
>>>
xmlrpclib.loads('<params>\n<param>\n<value><dateTime.iso8601>18500101T00:00:00</dateTime.iso8601></value>\n</param>\n</params>\n',
use_datetime=True)
((datetime.datetime(1850, 1, 1),), None)

Dumping however doesn't work:
>>> xmlrpclib.dumps((datetime.datetime(1850, 1, 1),))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/xmlrpclib.py", line 1080, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.5/xmlrpclib.py", line 623, in dumps
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 635, in __dump
    f(self, value, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 725, in dump_datetime
    write(value.strftime("%Y%m%dT%H:%M:%S"))
  File "datetime.py", line 791, in strftime
    return _wrap_strftime(self, fmt, self.timetuple())
  File "datetime.py", line 181, in _wrap_strftime
    "methods require year >= 1900" % year)
ValueError: year=1850 is before 1900; the datetime strftime() methods
require year >= 1900

This ValueError just shows an implementation detail.

Note that it's also possible to send and receive dates before 1900 using
xmlrpclib.DateTime objects.


> Number family (MJD or JDN) or Rata Die are more appropriate and much
> easier to use. I wish somebody could add both to the datetime module.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080402/1d4bf590/attachment.htm 

From tnelson at onresolve.com  Wed Apr  2 14:16:51 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 05:16:51 -0700
Subject: [Python-Dev] No time for svn merge
In-Reply-To: <47F34A85.5020909@cheimes.de>
References: <fsojc4$dm9$1@ger.gmane.org>	<47EFFD64.8020006@v.loewis.de>
	<1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com>
	<47F00C33.1050503@v.loewis.de>
	<5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com>
	<47F0113D.8020408@v.loewis.de>
	<5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com>
	<47F07068.3060503@v.loewis.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B6@EXMBX04.exchhosting.com>
	<47F34A85.5020909@cheimes.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381F9@EXMBX04.exchhosting.com>

Christian Heimes [mailto:lists at cheimes.de]:
> Trent Nelson schrieb:
> > I think holding a developer accountable for merging or blocking to
> py3k when they commit to trunk is a great idea.  Who better to pass
> judgement on such an activity than the person closest to it?
>
> Blocking a revision makes my job as The Merger easier.
>
> I'm not so sure about the merging part. It takes some experience with
> the Python 3.0 code base to know the subtle differences in the C API.
> Most merges are straight forward for me. If you enforce the forward
> merging on every developer it may slow down development.
> Each regular merge takes me about 45 minutes of computer time but less
> than 15 supervisor time. The computer time is mostly compile and test
> time in the background. If everybody merges her own code to 3.0 it
> still takes *everybody* about 10 minutes of time and 45 minutes of computer
> time.

Ah, right, I wasn't thinking about the implication of code affecting the C base for some reason, but that's entirely reasonable.  Perhaps each developer should be accountable for either:

a) blocking
b) merging, if they're able to do so
c) if they're unable to merge, replying to the relevant python-checkins@ e-mail indicating that they're unable to handle trunk -> py3k for whatever reason (e.g. not familiar with py3k code base)

Other developers could then pitch in and help merge if someone requests it via e-mail.  I'd think that would make The Merger's life easier.

        Trent.

From tnelson at onresolve.com  Wed Apr  2 18:20:21 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 09:20:21 -0700
Subject: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>

Looking into some of the recent Windows buildbot failures, I see things like this:

sqlite3 : error PRJ0008 : Could not delete file 'c:\buildbot\trunk.heller-windows-amd64\build\PCbuild\amd64\sqlite3_d.dll'.

build-amd64.bat doesn't go through the kill_python.c hoopla, so I figure the above error is being caused by the fact that an erroneous/stalled python_d.exe from a previous run is still open.  I was looking at modifying kill_python.c to accept an 'x64' argument if we want to kill amd64\python_d.exe instead of the usual 32-bit exe, however, this caught my attention:

if ((strstr(path, "pcbuild\\python_d.exe") != NULL) ||
    (strstr(path, "\\build\\python.exe") != NULL)) {
        printf("Terminating %s (pid %d)\n", path, pids[i]);
        if (!TerminateProcess(hProcess, 1)) {
                printf("Termination failed: %d\n", GetLastError());
                return 1;
        }
        return 0;

That'll kill the first python_d.exe instance it finds matching the given path; given that our buildbots run trunk/release25-maint/py3k in parallel, it seems as though it wouldn't be hard for us to get into a situation where kill_python.exe ends up killing the wrong python_d.exe (i.e. trunk checkin, trunk builds, starts testing, py3k checkin, calls kill_python.exe, kills trunk's python_d.exe that was in the middle of testing).

That can't be helping our cause, unless I'm missing something...  Unless anyone advises otherwise, I'll start on a patch.


        Trent.




From martin at v.loewis.de  Wed Apr  2 20:39:41 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2008 20:39:41 +0200
Subject: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our
 cause
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>
Message-ID: <47F3D2ED.4070109@v.loewis.de>

> That'll kill the first python_d.exe instance it finds matching the
> given path; given that our buildbots run trunk/release25-maint/py3k
> in parallel

That's actually not a given: we currently *don't* run multiple builds
simultaneously on the same slave.

> Unless anyone advises otherwise, I'll start on a patch.

If you can make it less error-prone, sure, go ahead.

Regards,
Martin


From guido at python.org  Wed Apr  2 20:47:02 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Apr 2008 11:47:02 -0700
Subject: [Python-Dev] Thread-safe file objects, the return
In-Reply-To: <loom.20080402T100849-143@post.gmane.org>
References: <loom.20080331T194117-885@post.gmane.org>
	<ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>
	<loom.20080402T100849-143@post.gmane.org>
Message-ID: <ca471dc20804021147h77d89972wb5d1726514433637@mail.gmail.com>

On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>  Guido van Rossum <guido <at> python.org> writes:
>  > Your solution (a counter) seems fine except I think perhaps the
>  > close() call should not raise IOError -- instead, it should set a flag
>  > so that the thread that makes the counter go to zero can close the
>  > thread (after all the file got closed while it was being used).
>
>  I agree with Gregory: we should be explicit about what happens. I wonder
>  what we would gain from that approach - apart from encouraging dangerous
>  coding practices :)
>  It also depends how far we want to go: I am merely proposing to fix the
>  crashes, do we want to provide a "smarter" close() variation that does what
>  you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose().

I think that for other (non-closing) operations we should be fine,
given the Posix requirement that streams have an internal lock. While
multiple threads reading from a file sounds insane, multiple files
writing to a file is pretty common (think of stdout or stderr) and
should be supported.

>  > There are of course other concurrency issues besides close -- what if
>  > two threads both try to do I/O on the file? What will the C stdio
>  > library do in that case? Are stdio files thread-safe at the C level?
>
>  According to the glibc documentation, at
>  http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html :
>
>  ? The POSIX standard requires that by default the stream operations are
>  atomic. I.e., issuing two stream operations for the same stream in two
>  threads at the same time will cause the operations to be executed as if
>  they were issued sequentially. The buffer operations performed while
>  reading or writing are protected from other uses of the same stream. To do
>  this each stream has an internal lock object which has to be (implicitly)
>  acquired before any work can be done. ?
>
>  So according to POSIX rules it should be perfectly safe.

Cool.

>  In any case, someone would have to try my patch under Windows and OS X and
>  see if test_file.py passes without crashing.

I know Windows has internal locks on stdio. I trust that OSX, being a
BSD descendant, is posix compliant. So I'm not worried about these.

+1 on your patch, assuming some other developer reviews it and submits it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Apr  2 20:59:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2008 20:59:40 +0200
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F34E34.9010507@gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>
	<47F34E34.9010507@gmail.com>
Message-ID: <47F3D79C.3020307@v.loewis.de>

> Personally, I've never really understood the purpose of 
> test_support.TESTFN. Whenever I've needed a temporary file for a test, I 
> just use the tempfile module (e.g. test_cmd_line_script, test_runpy). 
> Tests using that module don't care if the old files take 'a while' to 
> get deleted on Windows, as tempfile uses a different name each time anyway.
> 
> Is using a fixed TESTFN just an old approach that predates the existence 
> of a robust tempfile module in the standard library?

No. I believe the rationale for TESTFN is to provide a fixed name,
precisely so that the test suite doesn't leave tons of garbage around.

Regards,
Martin

From tnelson at onresolve.com  Wed Apr  2 21:12:17 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 2 Apr 2008 12:12:17 -0700
Subject: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our
 cause
In-Reply-To: <47F3D2ED.4070109@v.loewis.de>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>
	<47F3D2ED.4070109@v.loewis.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17F1F2EC@EXMBX04.exchhosting.com>

> > That'll kill the first python_d.exe instance it finds matching the
> > given path; given that our buildbots run trunk/release25-maint/py3k
> > in parallel
>
> That's actually not a given: we currently *don't* run multiple builds
> simultaneously on the same slave.

I thought the slave lock only applied per branch, not per host?

> > Unless anyone advises otherwise, I'll start on a patch.
>
> If you can make it less error-prone, sure, go ahead.

Spent a bit of time on it this evening; as it happens, in order to enumerate 64-bit processes, you need to be a 64-bit process yourself.  As it's much easier managing 32-bit vs. x64 binaries when they're a .vcproj part of pcbuild.sln, I'm looking into adding kill_python as a .vcproj and configure the solution such that it builds and runs this before any other project.  That'll automatically take care of choosing the right version to run depending on whether 'Win32' or 'x64' is selected as the platform.  It'll also simplify the verification logic that checks if it's the right python_d.exe -- the path of the .exe needs to match the path of the running kill_python.exe.

        Trent.

From martin at v.loewis.de  Wed Apr  2 21:29:02 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Apr 2008 21:29:02 +0200
Subject: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our
 cause
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17F1F2EC@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>
	<47F3D2ED.4070109@v.loewis.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17F1F2EC@EXMBX04.exchhosting.com>
Message-ID: <47F3DE7E.7070808@v.loewis.de>

Trent Nelson wrote:
>>> That'll kill the first python_d.exe instance it finds matching the
>>> given path; given that our buildbots run trunk/release25-maint/py3k
>>> in parallel
>> That's actually not a given: we currently *don't* run multiple builds
>> simultaneously on the same slave.
> 
> I thought the slave lock only applied per branch, not per host?

As the name suggests, it's per slave :-) a slave being the name/password
pair, along with the buildbot.tac file.

The sequencing of builds per builder is not a locking mechanism,
actually; a single builder just won't schedule a new build as long
as a build is already running.

So we have currently three builders per slave, and these are serialized
with the slave lock (hence the occasional long sequences of yellow
"Build nnn": the build starts, tries to acquire the slave lock, which
then a different builder still holds).

Now, on your machine, I understand you also have two slaves running.
They might, in principle, kill each other's python_d.exe processes,
except that the paths will be different there (amd64 vs. not).

Regards,
Martin

From priyarp.tech at gmail.com  Wed Apr  2 22:12:04 2008
From: priyarp.tech at gmail.com (Pree Raj)
Date: Wed, 2 Apr 2008 13:12:04 -0700
Subject: [Python-Dev] Python on ThreadX
Message-ID: <8bb4faa80804021312p6713db38i32373f1cc20a3632@mail.gmail.com>

Hi,
I have an application that used python on linux.
I want to port it to run on ThreadX.
Can you tell me if this can be possible.
If yes, how can I get started.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080402/07a93647/attachment.htm 

From alex.neundorf at kitware.com  Wed Apr  2 22:37:26 2008
From: alex.neundorf at kitware.com (Alexander Neundorf)
Date: Wed, 2 Apr 2008 22:37:26 +0200
Subject: [Python-Dev] Python on ThreadX
In-Reply-To: <8bb4faa80804021312p6713db38i32373f1cc20a3632@mail.gmail.com>
References: <8bb4faa80804021312p6713db38i32373f1cc20a3632@mail.gmail.com>
Message-ID: <806d41050804021337x7faff46ap4be31c14fd969006@mail.gmail.com>

On Wed, Apr 2, 2008 at 10:12 PM, Pree Raj <priyarp.tech at gmail.com> wrote:
> Hi,
> I have an application that used python on linux.
> I want to port it to run on ThreadX.
> Can you tell me if this can be possible.
> If yes, how can I get started.

I managed to get Python to work under eCos, which is also a small RTOS
(one binary firmware image, no shared libraries, one process space).
So I think it should be possible for ThreadX too.
I did this with the cmake-based build.

Alex

From musiccomposition at gmail.com  Wed Apr  2 22:55:13 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 2 Apr 2008 15:55:13 -0500
Subject: [Python-Dev] No time for svn merge
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381F9@EXMBX04.exchhosting.com>
References: <fsojc4$dm9$1@ger.gmane.org>
	<1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com>
	<47F00C33.1050503@v.loewis.de>
	<5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com>
	<47F0113D.8020408@v.loewis.de>
	<5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com>
	<47F07068.3060503@v.loewis.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381B6@EXMBX04.exchhosting.com>
	<47F34A85.5020909@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381F9@EXMBX04.exchhosting.com>
Message-ID: <1afaf6160804021355o8c97cf0le839b6623971dba8@mail.gmail.com>

On Wed, Apr 2, 2008 at 7:16 AM, Trent Nelson <tnelson at onresolve.com> wrote:

> Christian Heimes [mailto:lists at cheimes.de]:
> > Trent Nelson schrieb:
> > > I think holding a developer accountable for merging or blocking to
> > py3k when they commit to trunk is a great idea.  Who better to pass
> > judgement on such an activity than the person closest to it?
> >
> > Blocking a revision makes my job as The Merger easier.
> >
> > I'm not so sure about the merging part. It takes some experience with
> > the Python 3.0 code base to know the subtle differences in the C API.
> > Most merges are straight forward for me. If you enforce the forward
> > merging on every developer it may slow down development.
> > Each regular merge takes me about 45 minutes of computer time but less
> > than 15 supervisor time. The computer time is mostly compile and test
> > time in the background. If everybody merges her own code to 3.0 it
> > still takes *everybody* about 10 minutes of time and 45 minutes of
> computer
> > time.
>
> Ah, right, I wasn't thinking about the implication of code affecting the C
> base for some reason, but that's entirely reasonable.  Perhaps each
> developer should be accountable for either:
>
> a) blocking
> b) merging, if they're able to do so
> c) if they're unable to merge, replying to the relevant python-checkins@
> e-mail indicating that they're unable to handle trunk -> py3k for whatever
> reason (e.g. not familiar with py3k code base)
>
> Other developers could then pitch in and help merge if someone requests it
> via e-mail.  I'd think that would make The Merger's life easier.

I think we should let The Merger decide what makes his life easier. :)

>
>
>        Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080402/abc94f45/attachment.htm 

From scott+python-dev at scottdial.com  Wed Apr  2 22:58:03 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 02 Apr 2008 16:58:03 -0400
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F3D79C.3020307@v.loewis.de>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>	<47F34E34.9010507@gmail.com>
	<47F3D79C.3020307@v.loewis.de>
Message-ID: <47F3F35B.1010005@scottdial.com>

Martin v. L?wis wrote:
>> Is using a fixed TESTFN just an old approach that predates the existence 
>> of a robust tempfile module in the standard library?
> 
> No. I believe the rationale for TESTFN is to provide a fixed name,
> precisely so that the test suite doesn't leave tons of garbage around.

I figured this was going to be the answer, and I wonder if it would be 
better to use the tempfile module to generate unique filenames that are 
all contained in a special directory. It would seem functionally 
equivalent in the sense that removing the garbage is easy 
"rmtree(TESTDIR)", and it would also solve the windows problem of files 
that can't be unlinked. I suppose one would have to periodically 
clean-up their temp folder of the files left behind, but that's still an 
improvement over what is currently required if TESTFN is not unlinked.

The only reason I could see the switch being undesirable is if you 
wanted to go read the test file by hand, but I'm not sure anyone is 
doing that.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From barry at python.org  Thu Apr  3 00:00:09 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 2 Apr 2008 18:00:09 -0400
Subject: [Python-Dev] Building next alphas
Message-ID: <FDB62F0D-070F-4BD3-A4BF-9D3038EE556C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is a reminder that I am going to start building the next alpha  
releases for Python 2.6 and 3.0 now.  Please, no checkins unless you  
get approval from me, and until you hear that the freeze is lifted.

I am now on freenode #python-dev, IM, and Jabber if you need to  
contact me.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/QB6XEjvBPtnXfVAQLn8QQArynMsNeeb6gqjUCSYuupM2XXAbwP5XOX
LbTeGN+vM13uNK32fI47rDaPEfudfGnrd3Ttc1pg6/S/MOo5T41zs/TX2jdMEQ4g
6zCtk6xJiexGbExKioiTVdYgiqA8C6u+XY8aU2ogklD1h7kfEOWKw5urXkValFhG
Iymq6mrEyJQ=
=d/L3
-----END PGP SIGNATURE-----

From mhammond at skippinet.com.au  Thu Apr  3 00:50:10 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 3 Apr 2008 09:50:10 +1100
Subject: [Python-Dev] [Distutils] FW: [issue2513]
	64bit	cross	compilation on windows
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BA@EXMBX04.exchhosting.com>
References: <002401c892ff$a8254630$f86fd290$@com.au>	<00c801c893dd$32d03320$98709960$@com.au>	<47F2C163.9000402@v.loewis.de>
	<016001c89460$944f10c0$bced3240$@com.au>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BA@EXMBX04.exchhosting.com>
Message-ID: <022b01c89513$ef278140$cd7683c0$@com.au>

> I think this is fine; we don't really have a notion of compiling for a
> native platform, nor is the build machine's architecture factored into
> the equation.

Actually, I think it is slightly.  IIUC, the AMD64 build currently assumes
it can execute x86 executables in various places.  To fix this, the build
process for each of the platforms would be slightly different.  My proposal
would allow (eg) 'PCBuild\python.exe' or 'PCBuild\make_versioninfo.exe' to
specified in build processes, with the knowledge it will work, regardless of
what platform it is being run on.

For example, with my idea, you could write code such as:

  exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "python.exe"))
  popen(exe, ...)

and expect it to work on all platforms.  If I understand correctly, I now
need to write something like:

  if "AMD64" in sys.platform:
    exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "amd64",
"python.exe"))
  else if "32 bit" in sys.platform:
    exe = os.path.join(os.getenv("PYTHON_HOME"), "PCBuild", "python.exe"))
  else:
    # hrm - I obviously don?t know about new platforms yet...    
  popen(exe, ...)

In other words, I think there is real advantage to simple scripts knowing
that './python' or './PCBuild/python.exe' will always be executable (and
always give the "native" version if the platform does emulation)

But I accept I'm waving appendages in the wind, and without a concrete
use-case I will take Martin's advice and let it go.

Cheers,

Mark



From skip at pobox.com  Thu Apr  3 05:36:24 2008
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 2 Apr 2008 22:36:24 -0500
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
Message-ID: <18420.20664.231996.696687@montanaro-dyndns-org.local>

    Ralf> anyone care to take a look at:
    Ralf> http://bugs.python.org/issue2014
    Ralf> It's about xmlrpclib not being able to send datetime objects with
    Ralf> dates before 1900.

It's actually not xmlrpclib which has the limitation, but
datetime.strftime().  That's a known limitation.  Here's the comment in the
datetime code:

        /* Give up if the year is before 1900.
         * Python strftime() plays games with the year, and different
         * games depending on whether envar PYTHON2K is set.  This makes
         * years before 1900 a nightmare, even if the platform strftime
         * supports them (and not all do).
         * We could get a lot farther here by avoiding Python's strftime
         * wrapper and calling the C strftime() directly, but that isn't
         * an option in the Python implementation of this module.
         */

That, in turn, calls time.strftime() which calls the underlying system's
strftime() library function.

Personally, I don't think patching xmlrpclib is the right place to "fix"
this problem.  It's possible that the datetime comment is no longer correct
and that limitation should be reconsidered.  I see no other mention of
PYTHON2K in any .c, .h or .py files in the trunk.

Skip


From guido at python.org  Thu Apr  3 05:52:52 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Apr 2008 20:52:52 -0700
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <18420.20664.231996.696687@montanaro-dyndns-org.local>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>

On Wed, Apr 2, 2008 at 8:36 PM,  <skip at pobox.com> wrote:
>     Ralf> anyone care to take a look at:
>     Ralf> http://bugs.python.org/issue2014
>     Ralf> It's about xmlrpclib not being able to send datetime objects with
>     Ralf> dates before 1900.
>
>  It's actually not xmlrpclib which has the limitation, but
>  datetime.strftime().  That's a known limitation.  Here's the comment in the
>  datetime code:
>
>         /* Give up if the year is before 1900.
>          * Python strftime() plays games with the year, and different
>          * games depending on whether envar PYTHON2K is set.  This makes
>          * years before 1900 a nightmare, even if the platform strftime
>          * supports them (and not all do).
>          * We could get a lot farther here by avoiding Python's strftime
>          * wrapper and calling the C strftime() directly, but that isn't
>          * an option in the Python implementation of this module.
>          */
>
>  That, in turn, calls time.strftime() which calls the underlying system's
>  strftime() library function.
>
>  Personally, I don't think patching xmlrpclib is the right place to "fix"
>  this problem.  It's possible that the datetime comment is no longer correct
>  and that limitation should be reconsidered.  I see no other mention of
>  PYTHON2K in any .c, .h or .py files in the trunk.

I'd like to see this fixed if possible, but I'm not sure how -- the C
level 'struct tm' has (year - 1900) in the tm_year member, and I'm not
sure that implementations are required to do anything with negative
values there. We'd have to reimplement all of strftime ourselves. I'm
not against that, but there are higher priorities.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Apr  3 06:21:11 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 3 Apr 2008 00:21:11 -0400
Subject: [Python-Dev] Building next alphas
In-Reply-To: <FDB62F0D-070F-4BD3-A4BF-9D3038EE556C@python.org>
References: <FDB62F0D-070F-4BD3-A4BF-9D3038EE556C@python.org>
Message-ID: <5C3C4C91-88B9-49A7-924C-21ACB5225A46@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 2, 2008, at 6:00 PM, Barry Warsaw wrote:
>
> This is a reminder that I am going to start building the next alpha  
> releases for Python 2.6 and 3.0 now.  Please, no checkins unless you  
> get approval from me, and until you hear that the freeze is lifted.
>
> I am now on freenode #python-dev, IM, and Jabber if you need to  
> contact me.

I've been battling the flu and got distracted for a few hours tonight,  
so I'm not quite done with the releases.  However, I've tagged and  
tar'd 'em so it should just be a matter of uploading the files and  
updating the site.  I should finish tomorrow.

I'm thawing the trees, so you can go ahead and start committing things  
again, but /please/ be especially conservative over the next 24-48  
hours.  Make sure your changes don't break anything, just in case my  
virus-addled brain screwed something up and I need to cut another  
release.

- -Barry

P.S. Huge thanks to Benjamin Peterson, both for a quick last minute  
fix to the 3.0 NEWS file via IRC, and his wonderful release.py  
script.  I've hacked it up a bit, but it was exactly what I was  
looking for, and it made things go much smoother this time.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/RbOHEjvBPtnXfVAQKCOwQArrYXC9X3lOvqRTwWQ9SYPH+6n1VN4MrT
WNm+jhsbiwZq8EuNslCBW3/52HP/wM7jlYizKZCL+cbcFaevNhWjjbPtwSTkJjVy
/uKG/NcDYQsPH3n4mET3/XlF5JrfS51avLSD7YebucTph9+otzI8LkK0Unvdbtq+
/86m3lEZAlY=
=q005
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Thu Apr  3 08:41:44 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 03 Apr 2008 08:41:44 +0200
Subject: [Python-Dev] [Distutils] FW:
 [issue2513]	64bit	cross	compilation on windows
In-Reply-To: <022b01c89513$ef278140$cd7683c0$@com.au>
References: <002401c892ff$a8254630$f86fd290$@com.au>	<00c801c893dd$32d03320$98709960$@com.au>	<47F2C163.9000402@v.loewis.de>	<016001c89460$944f10c0$bced3240$@com.au>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E17E381BA@EXMBX04.exchhosting.com>
	<022b01c89513$ef278140$cd7683c0$@com.au>
Message-ID: <47F47C28.9090203@v.loewis.de>


> Actually, I think it is slightly.  IIUC, the AMD64 build currently assumes
> it can execute x86 executables in various places.  To fix this, the build
> process for each of the platforms would be slightly different.

Why does that need fixing? The AMD64 build *can* execute x86 binaries,
whether it is performed on Win32 or Win64.

A problem only arises if the AMD64 build assumes that it can execute
AMD64 binaries, and it is a cross-compilation. As a consequence, we
must never execute the python.exe that we just built during the build
process.

>  My proposal
> would allow (eg) 'PCBuild\python.exe' or 'PCBuild\make_versioninfo.exe' to
> specified in build processes, with the knowledge it will work, regardless of
> what platform it is being run on.

No to python.exe, yes to make_versioninfo.exe. make_versioninfo.exe is
*always* an x86 binary, so it is always safe to execute on Windows.
If you tried to execute PCBuild\python.exe, it would break my builds:
I don't *have* x86 binaries in PCBuild when I cross-compile for AMD64.

Instead, I've changed the build process to run HOST_PYTHON if that is
set, and only fall back to PCBuild\python.exe if it isn't.

Regards,
Martin

From schmir at gmail.com  Thu Apr  3 09:29:57 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Thu, 3 Apr 2008 09:29:57 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <18420.20664.231996.696687@montanaro-dyndns-org.local>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
Message-ID: <932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>

On Thu, Apr 3, 2008 at 5:36 AM, <skip at pobox.com> wrote:

>
> It's actually not xmlrpclib which has the limitation, but
> datetime.strftime().  That's a known limitation.  Here's the comment in
> the
> datetime code:
> [snip]
> Personally, I don't think patching xmlrpclib is the right place to "fix"
> this problem.  It's possible that the datetime comment is no longer
> correct
>

yes, you're right. but I didn't feel like writing a strftime implementation
(which has probably even less chance of being committed). This patch is
rather tiny, it's easy to understand and it works now.


> and that limitation should be reconsidered.  I see no other mention of
> PYTHON2K in any .c, .h or .py files in the trunk.
>
> Skip
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080403/ed8e1e56/attachment.htm 

From schmir at gmail.com  Thu Apr  3 09:34:27 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Thu, 3 Apr 2008 09:34:27 +0200
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>
Message-ID: <932f8baf0804030034g625d877ej7dd5f727ad41cc80@mail.gmail.com>

On Thu, Apr 3, 2008 at 5:52 AM, Guido van Rossum <guido at python.org> wrote:

>
> I'd like to see this fixed if possible, but I'm not sure how -- the C
> level 'struct tm' has (year - 1900) in the tm_year member, and I'm not
> sure that implementations are required to do anything with negative
> values there. We'd have to reimplement all of strftime ourselves. I'm
> not against that, but there are higher priorities.
>

Have you considered using the pure python datetime implementation from the
pypy project for py3k?
It's even based on your own code :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080403/51c43b85/attachment.htm 

From heshan.suri at gmail.com  Thu Apr  3 13:47:11 2008
From: heshan.suri at gmail.com (Heshan Suriyaarachchi)
Date: Thu, 3 Apr 2008 17:17:11 +0530
Subject: [Python-Dev] Annotations support in python
Message-ID: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>

 Hi,
     I need to work with annotations as it is said in [1]. Currently I am
using python 2.5.1. I would like to know whether the
next release of python will support this feature. If the next version
support this feature I would like to know when are you
planning to release it. I used the __future__ module but I could not get the
annotations with it.

[1] -  http://www.python.org/dev/peps/pep-3107/

Thanx
Heshan Suriyaarachchi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080403/0c2792e4/attachment.htm 

From thomas at python.org  Thu Apr  3 14:20:48 2008
From: thomas at python.org (Thomas Wouters)
Date: Thu, 3 Apr 2008 14:20:48 +0200
Subject: [Python-Dev] Annotations support in python
In-Reply-To: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>
References: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>
Message-ID: <9e804ac0804030520i4c3201r51f72ff1a26d3314@mail.gmail.com>

On Thu, Apr 3, 2008 at 1:47 PM, Heshan Suriyaarachchi <heshan.suri at gmail.com>
wrote:

>  Hi,
>      I need to work with annotations as it is said in [1]. Currently I am
> using python 2.5.1. I would like to know whether the
> next release of python will support this feature. If the next version
> support this feature I would like to know when are you
> planning to release it. I used the __future__ module but I could not get
> the annotations with it.
>
> [1] -  http://www.python.org/dev/peps/pep-3107/
>


The 3xxx PEPs are all about Python 3.0. Python 3.0 is the version that will
support function annotations. Python 2.6 will almost certainly not, and 2.5
will most assuredly not -- features don't get added after the major version
has been released, we only fix bugs.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080403/e7dc8204/attachment.htm 

From mail at timgolden.me.uk  Thu Apr  3 15:39:47 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 03 Apr 2008 14:39:47 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F3592D.3080700@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk>
Message-ID: <47F4DE23.3020202@timgolden.me.uk>

[re tests which fail because something's holding a file
open with SHARE_DELETE]

Well I've tried my best, but I can't come up with a solution
which guarantees to overcome this obstacle. I set up a fairly
aggressive directory watcher which, when it sees a test file
being created, takes a SHARE_DELETE handle on it and releases
it immediately. (Guessing that this is what the other apps
are doing).

Running the test suite from HEAD, this generates all manner
of access-denied type errors as per the original output.

I used tempfile to generate a random TESTFN in the current directory
rather than the static @test. And implemented the rename-shim
discussed previously, renaming to a different random name, courtesy
of mktemp. With those in place, most tests run without error. But
I'm still getting errors in the same sort of areas which Steven B
originally reported.

The one error I can't see a way round is the access denied (which
manifests as Permission Error) which is the documented result of
attempting to open a file with a pending delete -- ie the delete
succeeded but hasn't completed yet.

An additional complication is that there are hundreds of
instances throughout the tests where the test simply calls
os.unlink/os.remove to undo the test file. To have some more
robust central deletion I had to go through and update 68 tests.

I'll keep trying, but in the current state I'm not convinced
the situation's improved enough for me to bother uploading
a patch.

TJG

From tnelson at onresolve.com  Thu Apr  3 19:44:51 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 3 Apr 2008 10:44:51 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F4DE23.3020202@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk>,<47F4DE23.3020202@timgolden.me.uk>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>

[Disclaimer: thought dump e-mail, signal to noise ratio may be subpar.]

Sounds like you're at least making steps forward in the right direction, despite the activity probably being quite disheartening.  Based on what you've said below and the rest of the conversation, here are my thoughts for an approach:

1.  For a given python[_d].exe, always use the same test directory, but hash it against the entire python process path such that it's unique only for a given python instance.
2.  Make sure every time a test wants a temp file, it gets a unique one in this directory.  Sounds like your TESTFN modification would take care of this for those tests using TESTFN; if TESTFN is the preferred approach then any other tests using tempfile or hardcoding file names would then be changed to use this instead.
3.  In order to address tests that either call the test_support methods for removing files/directories, or those that call os.(unlink|remove), do what ever needs to be done to make these no-ops on Windows if an error occurs.
4.  At the end of the regrtest.py run, create a suspended arbitrary process (i.e. explorer.exe), hijack the main thread context of the process and inject a routine (i.e. overwrite the thread context's instruction pointers) that takes care of removing the temporary directory that was used for testing -- patiently re-trying if any failures occur until all rogue processes also accessing the file(s) stop doing so.  Resume the thread before exiting python.

Heh.  Sounds crazy?  It did to me as well, until I came across http://www.catch22.net/tuts/selfdel.asp, which documents the approach.  It's not particularly necessary in our case, we could simply spawn another python process at the end of regrtest.py that patiently attempts to remove the test directory we just used when the python process that was executing regrtest.py exits.

We could then modify regrtest.py such that it will use the same approach if the hashed test directory already exists at the start of a run and it can't remove it via os.unlink.  If we *still* run into issues here on the buildbots, say if regrtest.py blocks on our helper process, which for the life of it can't remove some/all of the test files -- it'd be interesting to write something that grok's all open handles for all processes and attempts to figure out what it is that keeps these files open -- i.e. same thing that procexp.exe does when you search for a handle.

Or, keeping it simple, rather than a separate process and hashed test directory based on python process path, just have a common directory, i.e. %TEMP%\python-regrtest, and use an incrementing sequence number for each test run's test directory, i.e. if there are directories 001, 002 and 003 in the temp dir, when regrtest.py starts, it'll try delete all of these -- if it can't (which is what we'd want if another test is already running), it adds 1 to the highest numbered directory it couldn't delete.

Guess it all depends on how much effort we want to put into cleaning up our test directory really -- just ensuring tests get a clean area and unique file names each run is the easy part.

    Trent.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Tim Golden [mail at timgolden.me.uk]
Sent: 03 April 2008 09:39
To: Python-Dev
Subject: Re: [Python-Dev] fixing tests on windows

[re tests which fail because something's holding a file
open with SHARE_DELETE]

Well I've tried my best, but I can't come up with a solution
which guarantees to overcome this obstacle. I set up a fairly
aggressive directory watcher which, when it sees a test file
being created, takes a SHARE_DELETE handle on it and releases
it immediately. (Guessing that this is what the other apps
are doing).

Running the test suite from HEAD, this generates all manner
of access-denied type errors as per the original output.

I used tempfile to generate a random TESTFN in the current directory
rather than the static @test. And implemented the rename-shim
discussed previously, renaming to a different random name, courtesy
of mktemp. With those in place, most tests run without error. But
I'm still getting errors in the same sort of areas which Steven B
originally reported.

The one error I can't see a way round is the access denied (which
manifests as Permission Error) which is the documented result of
attempting to open a file with a pending delete -- ie the delete
succeeded but hasn't completed yet.

An additional complication is that there are hundreds of
instances throughout the tests where the test simply calls
os.unlink/os.remove to undo the test file. To have some more
robust central deletion I had to go through and update 68 tests.

I'll keep trying, but in the current state I'm not convinced
the situation's improved enough for me to bother uploading
a patch.

TJG
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From guido at python.org  Thu Apr  3 20:17:06 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Apr 2008 11:17:06 -0700
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
Message-ID: <ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>

On Thu, Apr 3, 2008 at 12:29 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> yes, you're right. but I didn't feel like writing a strftime implementation
> (which has probably even less chance of being committed). This patch is
> rather tiny, it's easy to understand and it works now.

Thinking about it more, given the slim chances that we'll reimplement
strftime, I think it's okay to fix this for xmlrpc specifically.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr  3 20:17:48 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Apr 2008 11:17:48 -0700
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <932f8baf0804030034g625d877ej7dd5f727ad41cc80@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>
	<932f8baf0804030034g625d877ej7dd5f727ad41cc80@mail.gmail.com>
Message-ID: <ca471dc20804031117p37443804pada49c01221e3496@mail.gmail.com>

On Thu, Apr 3, 2008 at 12:34 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> Have you considered using the pure python datetime implementation from the
> pypy project for py3k?

I wouldn't dream of it. datetime is considered performance critical by many.

> It's even based on your own code :)

Which was meant as a prototype only.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tnelson at onresolve.com  Thu Apr  3 20:33:23 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 3 Apr 2008 11:33:23 -0700
Subject: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our
 cause
In-Reply-To: <47F3D2ED.4070109@v.loewis.de>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E17E38501@EXMBX04.exchhosting.com>,
	<47F3D2ED.4070109@v.loewis.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27F@EXMBX04.exchhosting.com>

Committed new version of kill_python to trunk in r62129.

   Trent.
________________________________________
From: "Martin v. L?wis" [martin at v.loewis.de]
Sent: 02 April 2008 14:39
To: Trent Nelson
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Tools\buildbot\kill_python.c can't be helping our cause

> That'll kill the first python_d.exe instance it finds matching the
> given path; given that our buildbots run trunk/release25-maint/py3k
> in parallel

That's actually not a given: we currently *don't* run multiple builds
simultaneously on the same slave.

> Unless anyone advises otherwise, I'll start on a patch.

If you can make it less error-prone, sure, go ahead.

Regards,
Martin

From tjreedy at udel.edu  Thu Apr  3 23:09:35 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 3 Apr 2008 17:09:35 -0400
Subject: [Python-Dev] fixing tests on windows
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk><47F3592D.3080700@timgolden.me.uk>
	<47F4DE23.3020202@timgolden.me.uk>
Message-ID: <ft3h2b$ftl$1@ger.gmane.org>


"Tim Golden" <mail at timgolden.me.uk> wrote in message 
news:47F4DE23.3020202 at timgolden.me.uk...
| [re tests which fail because something's holding a file
| open with SHARE_DELETE]

There are a couple of things one can do in a directory's Properties box 
(right click) to reduce interference.
1. Under General/Advanced, uncheck 'allow Indexing Service' and click Yes 
to recursively apply to subdirs and files.
2. Under Sharing, check 'Make private'.  This is recursive, but only works 
for dirs under My Documents or on Desktop.  This would be most useful, I 
think, if buildbots run as a separate user.
Have these already been done?

tjr




From tnelson at onresolve.com  Thu Apr  3 23:40:04 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 3 Apr 2008 14:40:04 -0700
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB280@EXMBX04.exchhosting.com>

I started looking into this:

http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/289/step-test/0

Pertinent part:

test_asyncore
<snip>
test_asynchat

command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1
remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
]

I tried to replicate it on the buildbot in order to debug, which, surprisingly, I could do consistently by just running rt.bat -q -d -uall test_asynchat.  As the log above indicates, the python process becomes completely and utterly wedged, to the point that I can't even attach a remote debugger and step into it.

Digging further, I noticed that if I ran the following code in two different python consoles, EADDRINUSE was *NOT* being raised by socket.bind():

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('127.0.0.1', 54322))

However, take out the setsockopt line, and wallah, the second s.bind() will raise EADDRINUSE, as expected.  This manifests into a really bizarre issue with test_asynchat in particualr, as subsequent sock.accept() calls on the socket put python into the uber wedged state (can't even ctrl-c out at the console, need to kill the process directly).

Have to leave the office and head home so I don't have any more time to look at it tonight -- just wanted to post here for others to mull over.

    Trent.

From steven.bethard at gmail.com  Thu Apr  3 23:52:39 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 3 Apr 2008 15:52:39 -0600
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <ft3h2b$ftl$1@ger.gmane.org>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk> <47F3592D.3080700@timgolden.me.uk>
	<47F4DE23.3020202@timgolden.me.uk> <ft3h2b$ftl$1@ger.gmane.org>
Message-ID: <d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>

On Thu, Apr 3, 2008 at 3:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>  "Tim Golden" <mail at timgolden.me.uk> wrote in message
>  news:47F4DE23.3020202 at timgolden.me.uk...
>
> | [re tests which fail because something's holding a file
>  | open with SHARE_DELETE]
>
>  There are a couple of things one can do in a directory's Properties box
>  (right click) to reduce interference.
>  1. Under General/Advanced, uncheck 'allow Indexing Service' and click Yes
>  to recursively apply to subdirs and files.
>  2. Under Sharing, check 'Make private'.  This is recursive, but only works
>  for dirs under My Documents or on Desktop.  This would be most useful, I
>  think, if buildbots run as a separate user.
>  Have these already been done?

We already know how to stop the errors from arising by modifying the
Windows environment. In my particular case, it was sufficient to
disable Windows Search Service and the TortoiseSVN Image Overlays.
However, it would be *really* nice if the Python test suite just
worked out of the box, and you didn't have to change your normal
Windows environment.

Or were you suggesting that there is some programmatic way for the
test suite to create directories that disallow the Search Service,
etc.?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From mail at timgolden.me.uk  Thu Apr  3 23:58:24 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 03 Apr 2008 22:58:24 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <ft3h2b$ftl$1@ger.gmane.org>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk><47F3592D.3080700@timgolden.me.uk>	<47F4DE23.3020202@timgolden.me.uk>
	<ft3h2b$ftl$1@ger.gmane.org>
Message-ID: <47F55300.4060603@timgolden.me.uk>

Terry Reedy wrote:
> "Tim Golden" <mail at timgolden.me.uk> wrote in message 
> news:47F4DE23.3020202 at timgolden.me.uk...
> | [re tests which fail because something's holding a file
> | open with SHARE_DELETE]
> 
> There are a couple of things one can do in a directory's Properties box 
> (right click) to reduce interference.
> 1. Under General/Advanced, uncheck 'allow Indexing Service' and click Yes 
> to recursively apply to subdirs and files.
> 2. Under Sharing, check 'Make private'.  This is recursive, but only works 
> for dirs under My Documents or on Desktop.  This would be most useful, I 
> think, if buildbots run as a separate user.
> Have these already been done?

I realise that there are options on any given machine
to reduce the interference of known applications. (And
I assume that the people who own buildbots are in a
position to do what they can in that direction). I was
working more towards a situation where they wouldn't
have to, where the test suite could confidently be run
without having to allow for artefacts of this type.

Still trying: some new ideas from Trent elsewhere in
this thread which I'll look into tomorrow.

TJG

From tjreedy at udel.edu  Fri Apr  4 02:34:18 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 3 Apr 2008 20:34:18 -0400
Subject: [Python-Dev] fixing tests on windows
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com><d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com><47F252CE.1020702@timgolden.me.uk>
	<47F25F2E.4070408@timgolden.me.uk><52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com><47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk><47F4DE23.3020202@timgolden.me.uk>
	<ft3h2b$ftl$1@ger.gmane.org>
	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
Message-ID: <ft3t26$j8a$1@ger.gmane.org>


"Steven Bethard" <steven.bethard at gmail.com> wrote in message 
news:d11dcfba0804031452m911e1f3w4ec68a5967d69656 at mail.gmail.com...
| On Thu, Apr 3, 2008 at 3:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:
| >  "Tim Golden" <mail at timgolden.me.uk> wrote in message
| >  news:47F4DE23.3020202 at timgolden.me.uk...
| >
| > | [re tests which fail because something's holding a file
| >  | open with SHARE_DELETE]
| >
| >  There are a couple of things one can do in a directory's Properties 
box
| >  (right click) to reduce interference.
| >  1. Under General/Advanced, uncheck 'allow Indexing Service' and click 
Yes
| >  to recursively apply to subdirs and files.
| >  2. Under Sharing, check 'Make private'.  This is recursive, but only 
works
| >  for dirs under My Documents or on Desktop.  This would be most useful, 
I
| >  think, if buildbots run as a separate user.
| >  Have these already been done?
|
| We already know how to stop the errors from arising by modifying the
| Windows environment. In my particular case, it was sufficient to
| disable Windows Search Service and the TortoiseSVN Image Overlays.
| However, it would be *really* nice if the Python test suite just
| worked out of the box, and you didn't have to change your normal
| Windows environment.

If the testdir disallows the search indexer, then there should be no need 
to disable Windows Search Service.  If privatizing the dir kept other 
programs out, then likewise.

| Or were you suggesting that there is some programmatic way for the
| test suite to create directories that disallow the Search Service,
| etc.?

I suspect, but do not know, that the dialog box effects changes through 
user-programmable interfaces.  So while I would start with manual changes 
to see if that solves the problem, I presume there must be a system call 
for changing dir attributes.

tjr




From tonynelson at georgeanelson.com  Fri Apr  4 02:22:07 2008
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Thu, 3 Apr 2008 20:22:07 -0400
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk> <47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk> <47F3592D.3080700@timgolden.me.uk>
	<47F4DE23.3020202@timgolden.me.uk> <ft3h2b$ftl$1@ger.gmane.org>
	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
Message-ID: <p04330100c41b2341d236@[192.168.123.162]>

At 3:52 PM -0600 4/3/08, Steven Bethard wrote:
>On Thu, Apr 3, 2008 at 3:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:
 ...
>Or were you suggesting that there is some programmatic way for the
>test suite to create directories that disallow the Search Service,
>etc.?

I'd think that files and directories created in the TEMP directory would
normally not be indexed on any OS, including MSWindows.  But this is just a
guess.
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From barry at python.org  Fri Apr  4 03:47:01 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 3 Apr 2008 21:47:01 -0400
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
Message-ID: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I'm  
happy to announce the second alpha release of Python 2.6, and the  
fourth alpha release of Python 3.0.

Please note that these are alpha releases, and as such are not  
suitable for production environments.  We continue to strive for a  
high degree of quality, but there are still some known problems and  
the feature sets have not been finalized.  These alphas are being  
released to solicit feedback and hopefully discover bugs, as well as  
allowing you to determine how changes in 2.6 and 3.0 might impact  
you.  If you find things broken or incorrect, please submit a bug  
report at

    http://bugs.python.org

For more information and downloadable distributions, see the Python  
2.6 web
site:

    http://www.python.org/download/releases/2.6/

and the Python 3.0 web site:

    http://www.python.org/download/releases/3.0/

We are planning one more alpha release of each version, followed by  
two beta releases, with the final releases planned for August 2008.   
See PEP 361 for release details:

     http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
barry at python.org
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/WImHEjvBPtnXfVAQJmoQP+MzqNDI+Xt8zua/FE7Ca4TVXoIIy2uoOm
I1i3+vmevZ9vtAb9hcGwfEgPY4LSwb9Js4KnJJWMPaMuFJK4NgGoiMdj+t42zDbQ
bEzfBUOCoVkejLRxIQnWeJf1Hu8JocYyCHIRffv57/QdKpHuiSs8aE8GIT3STo3o
I88H5NY1GgI=
=WT2z
-----END PGP SIGNATURE-----

From skip at pobox.com  Fri Apr  4 05:48:06 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 3 Apr 2008 22:48:06 -0500
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
	<ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>
Message-ID: <18421.42230.77841.513592@montanaro-dyndns-org.local>


    Guido> Thinking about it more, given the slim chances that we'll
    Guido> reimplement strftime, I think it's okay to fix this for xmlrpc
    Guido> specifically.

Is there some reason we can't incorporate a suitable open source
implementation of strftime into the Python core?  Here's one example that
might provide a suitable starting point (BSD 4.4, then Tcl, then Apple
heritage):

    http://www.opensource.apple.com/darwinsource/WWDC2003/tcl-10/tcl/compat/strftime.c

It would probably need some tweaks for Windows, and some changes to the
macro usage.  Then there is the 1900 base year issue.  I'm certainly no
licensing expert, but it seems like we should be able to redistribute it.

Skip

From foom at fuhm.net  Fri Apr  4 05:28:51 2008
From: foom at fuhm.net (James Y Knight)
Date: Thu, 3 Apr 2008 23:28:51 -0400
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<ca471dc20804022052wb7443dfs5bf67aa2587870eb@mail.gmail.com>
Message-ID: <5F06CBDC-6DF2-4621-99FC-0087D809E471@fuhm.net>

On Apr 2, 2008, at 11:52 PM, Guido van Rossum wrote:
> I'd like to see this fixed if possible, but I'm not sure how -- the C
> level 'struct tm' has (year - 1900) in the tm_year member, and I'm not
> sure that implementations are required to do anything with negative
> values there. We'd have to reimplement all of strftime ourselves. I'm
> not against that, but there are higher priorities.

At least on linux, negative values work great. I have no idea if  
that's required by the spec or not. Someone would have to test on  
other platforms I guess...

James


From guido at python.org  Fri Apr  4 05:59:13 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 3 Apr 2008 20:59:13 -0700
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <18421.42230.77841.513592@montanaro-dyndns-org.local>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
	<ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>
	<18421.42230.77841.513592@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804032059w266be0b3l4cc928932659c70c@mail.gmail.com>

Someone else will have to do a thorough code review. Last time we got
something off the web it turned out to be awful (the float formatting
code -- I'm still reeling from that one).

On Thu, Apr 3, 2008 at 8:48 PM,  <skip at pobox.com> wrote:
>
>     Guido> Thinking about it more, given the slim chances that we'll
>     Guido> reimplement strftime, I think it's okay to fix this for xmlrpc
>     Guido> specifically.
>
>  Is there some reason we can't incorporate a suitable open source
>  implementation of strftime into the Python core?  Here's one example that
>  might provide a suitable starting point (BSD 4.4, then Tcl, then Apple
>  heritage):
>
>     http://www.opensource.apple.com/darwinsource/WWDC2003/tcl-10/tcl/compat/strftime.c
>
>  It would probably need some tweaks for Windows, and some changes to the
>  macro usage.  Then there is the 1900 base year issue.  I'm certainly no
>  licensing expert, but it seems like we should be able to redistribute it.
>
>  Skip
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Fri Apr  4 06:23:00 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 3 Apr 2008 23:23:00 -0500
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <ca471dc20804032059w266be0b3l4cc928932659c70c@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
	<ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>
	<18421.42230.77841.513592@montanaro-dyndns-org.local>
	<ca471dc20804032059w266be0b3l4cc928932659c70c@mail.gmail.com>
Message-ID: <18421.44324.892060.577502@montanaro-dyndns-org.local>


    Guido> Someone else will have to do a thorough code review. Last time we
    Guido> got something off the web it turned out to be awful (the float
    Guido> formatting code -- I'm still reeling from that one).

This isn't some oddball weekend project from an out-of-work programmer.
It's derived from BSD 4.4 and then looks like Tcl absorbed it as their
implementation of strftime.  The version I posted looks like it's part of
Darwin.  Should be fairly well wrung out.  I downloaded it and with a couple
mods I was able to use it as a substitute for strftime() in timemodule.c.
It compiles cleanly and seems to pass all tests.

My thought would be to only use this for Python 3.0.

Skip

From skip at pobox.com  Fri Apr  4 06:30:05 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 3 Apr 2008 23:30:05 -0500
Subject: [Python-Dev] xmlrpclib and dates before 1900
In-Reply-To: <ca471dc20804032059w266be0b3l4cc928932659c70c@mail.gmail.com>
References: <932f8baf0804011345m27206411o2033498a72738149@mail.gmail.com>
	<18420.20664.231996.696687@montanaro-dyndns-org.local>
	<932f8baf0804030029y262d7cfeicc22b5d94b7ffc08@mail.gmail.com>
	<ca471dc20804031117q758a8ad4teed7eb8e59eb9fc5@mail.gmail.com>
	<18421.42230.77841.513592@montanaro-dyndns-org.local>
	<ca471dc20804032059w266be0b3l4cc928932659c70c@mail.gmail.com>
Message-ID: <18421.44749.206803.852140@montanaro-dyndns-org.local>

> I downloaded it and with a couple mods I was able to use it as a
> substitute for strftime() in timemodule.c.  It compiles cleanly and seems
> to pass all tests.

I misspoke.  test_strptime fails with timezone issues.  That's probably just
my misunderstanding of how Python deals with timezones.  I make a quick
replacement of Tcl's tzname stuff to get it to compile, but the test fails.
It would probably not be much work for someone who understands how the
Python datetime/time code handles timezone names.

Skip


From nnorwitz at gmail.com  Fri Apr  4 06:49:22 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 3 Apr 2008 21:49:22 -0700
Subject: [Python-Dev] Annotations support in python
In-Reply-To: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>
References: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>
Message-ID: <ee2a432c0804032149i4bd493e5jed27aef793e6b53@mail.gmail.com>

The release schedule for 2.6/3.0 is http://www.python.org/dev/peps/pep-0361/
3.0 will have the feature, 2.6 may or may not.

n

On Thu, Apr 3, 2008 at 4:47 AM, Heshan Suriyaarachchi
<heshan.suri at gmail.com> wrote:
>  Hi,
>      I need to work with annotations as it is said in [1]. Currently I am
> using python 2.5.1. I would like to know whether the
> next release of python will support this feature. If the next version
> support this feature I would like to know when are you
>  planning to release it. I used the __future__ module but I could not get
> the annotations with it.
>
> [1] -  http://www.python.org/dev/peps/pep-3107/
>
> Thanx
> Heshan Suriyaarachchi
>
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>
>

From mail at timgolden.me.uk  Fri Apr  4 10:04:25 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 04 Apr 2008 09:04:25 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <ft3t26$j8a$1@ger.gmane.org>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com><d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com><47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk><52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com><47F33DE7.3030702@timgolden.me.uk>	<47F3592D.3080700@timgolden.me.uk><47F4DE23.3020202@timgolden.me.uk>	<ft3h2b$ftl$1@ger.gmane.org>	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
	<ft3t26$j8a$1@ger.gmane.org>
Message-ID: <47F5E109.4090608@timgolden.me.uk>

Terry Reedy wrote:
> If the testdir disallows the search indexer, then there should be no need 
> to disable Windows Search Service.  If privatizing the dir kept other 
> programs out, then likewise.
> 
> | Or were you suggesting that there is some programmatic way for the
> | test suite to create directories that disallow the Search Service,
> | etc.?
> 
> I suspect, but do not know, that the dialog box effects changes through 
> user-programmable interfaces.  So while I would start with manual changes 
> to see if that solves the problem, I presume there must be a system call 
> for changing dir attributes.

The problem is, I think, that it isn't just the Indexing Service
which generates this issue. TortoiseSVN is well known for doing
the same thing, and there could be now and potentially will be
in the future other programs. I don't think that hunting down
and turning off their interference case by case is a viable
solution in the long-term. Although it would obviously be a
way forward in the short term, _faute de mieux_.

[Tony Nelson]
> I'd think that files and directories created in the TEMP
> directory would normally not be indexed on any OS, including
> MSWindows. But this is just a guess.

I'm inclined to think you're right. And a first pass I had at
producing a solution simply used tempfile to do everything.
Unfortunately that's far more invasive than I was really
comfortable with at the time: at the very least, you have
to patch several tests which fail if there's an (escaped)
backslash in the path.

However, it's clear that my attempt to cause the minimum damage
isn't enough to clear the problem 100%. So I think the next move
is indeed to turn test_support.TESTFN into a function (in some
way) which generates a unique tempfile reference, possibly with
a context manager to clean up. Or something.

The complication is that, while most test simply want a handy
file to exist or be written to, and don't really care what
happens afterwards, some tests are testing the very mechanism
of creating/deleting a file etc. So a wholesale replacement
isn't necessarily straightforward.

On we go.

TJG


From mail at timgolden.me.uk  Fri Apr  4 10:04:25 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 04 Apr 2008 09:04:25 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <ft3t26$j8a$1@ger.gmane.org>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com><d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com><47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk><52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com><47F33DE7.3030702@timgolden.me.uk>	<47F3592D.3080700@timgolden.me.uk><47F4DE23.3020202@timgolden.me.uk>	<ft3h2b$ftl$1@ger.gmane.org>	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
	<ft3t26$j8a$1@ger.gmane.org>
Message-ID: <47F5E109.4090608@timgolden.me.uk>

<<< No Message Collected >>>

From mail at timgolden.me.uk  Fri Apr  4 10:29:32 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 04 Apr 2008 09:29:32 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>	<47F3592D.3080700@timgolden.me.uk>,
	<47F4DE23.3020202@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>
Message-ID: <47F5E6EC.6030402@timgolden.me.uk>

Trent Nelson wrote:
> 1.  For a given python[_d].exe, always use the same test directory, 
> but hash it against the entire python process path such that it's 
> unique only for a given python instance.

This is to guard against several build runs in parallel, presumably?

> 2.  Make sure every time a test wants a temp file, it gets a 
> unique one in this directory.  Sounds like your TESTFN 
> modification would take care of this for those tests using 
> TESTFN

Not quite; at present TESTFN is constant for one set of test runs. 
It would need to be a function call to change for every test
within the run.

; if TESTFN is the preferred approach then any other 
> tests using tempfile or hardcoding file names would then be 
> changed to use this instead.

Part of the problem (for me): I don't know if it is "the preferred
approach", merely that it's there and widely but not universally
used and is the likely cause of the locking problems we're seeing.
If anything's rolling its own tempfile solution, I'm inclined to
leave that alone. As someone's pointed out elsewhere, the temp
directories are almost certainly not monitored.

> 3.  In order to address tests that either call the 
> test_support methods for removing files/directories, 
> or those that call os.(unlink|remove), do what ever 
> needs to be done to make these no-ops on Windows if an 
> error occurs.

I'm reluctant to monkey patch os.(unlink|remove). I'd rather
provide and encourage the use of a test_support.remove_test_file
function, or even a context manager which did the same, failing
silently if needs be.

> 4.  At the end of the regrtest.py run, create a suspended arbitrary 
> process (i.e. explorer.exe), hijack the main thread context of the 
> process and inject a routine (i.e. overwrite the thread context's 
> instruction pointers) that takes care of removing the temporary 
> directory that was used for testing -- patiently re-trying if 
> any failures occur until all rogue processes also accessing the 
> file(s) stop doing so.  Resume the thread before exiting python.

OK. That was scary. (Interesting site, though).

[... snip ...]

> Guess it all depends on how much effort we want to put into cleaning 
> up our test directory really -- just ensuring tests get a clean area 
> and unique file names each run is the easy part.

Yes. I'm trying desperately hard to stick to a narrow remit of
getting tests to run consistently in the face of messy file-locking
semantics under Windows. I really don't want to wade into the minor
minefield of making all the tests run with consistent temp file
semantics. But I may have to.

TJG

From tnelson at onresolve.com  Fri Apr  4 10:29:16 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 4 Apr 2008 01:29:16 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F5E109.4090608@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com><d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com><47F252CE.1020702@timgolden.me.uk>
	<47F25F2E.4070408@timgolden.me.uk><52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com><47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk><47F4DE23.3020202@timgolden.me.uk>
	<ft3h2b$ftl$1@ger.gmane.org>
	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com>
	<ft3t26$j8a$1@ger.gmane.org>,<47F5E109.4090608@timgolden.me.uk>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB281@EXMBX04.exchhosting.com>

I agree with Tim, you can jump through as many hoops as you want (setting directories private, using %TEMP% exclusively, etc), but I doubt anything is going to change the behaviour of things like virus scanners, for example.

Tim, let me know if you need help with anything, perhaps we could set up a temporary branch outside of trunk to play around with various approaches to see what works best.  This'll ensure we don't adversely affect the main buildbots, but also give us the option to get different Windows boxes to build our test branch on demand.

    Trent.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Tim Golden [mail at timgolden.me.uk]
Sent: 04 April 2008 04:04
Cc: python-dev at python.org
Subject: Re: [Python-Dev] fixing tests on windows

Terry Reedy wrote:
> If the testdir disallows the search indexer, then there should be no need
> to disable Windows Search Service.  If privatizing the dir kept other
> programs out, then likewise.
>
> | Or were you suggesting that there is some programmatic way for the
> | test suite to create directories that disallow the Search Service,
> | etc.?
>
> I suspect, but do not know, that the dialog box effects changes through
> user-programmable interfaces.  So while I would start with manual changes
> to see if that solves the problem, I presume there must be a system call
> for changing dir attributes.

The problem is, I think, that it isn't just the Indexing Service
which generates this issue. TortoiseSVN is well known for doing
the same thing, and there could be now and potentially will be
in the future other programs. I don't think that hunting down
and turning off their interference case by case is a viable
solution in the long-term. Although it would obviously be a
way forward in the short term, _faute de mieux_.

[Tony Nelson]
> I'd think that files and directories created in the TEMP
> directory would normally not be indexed on any OS, including
> MSWindows. But this is just a guess.

I'm inclined to think you're right. And a first pass I had at
producing a solution simply used tempfile to do everything.
Unfortunately that's far more invasive than I was really
comfortable with at the time: at the very least, you have
to patch several tests which fail if there's an (escaped)
backslash in the path.

However, it's clear that my attempt to cause the minimum damage
isn't enough to clear the problem 100%. So I think the next move
is indeed to turn test_support.TESTFN into a function (in some
way) which generates a unique tempfile reference, possibly with
a context manager to clean up. Or something.

The complication is that, while most test simply want a handy
file to exist or be written to, and don't really care what
happens afterwards, some tests are testing the very mechanism
of creating/deleting a file etc. So a wholesale replacement
isn't necessarily straightforward.

On we go.

TJG

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From schmir at gmail.com  Fri Apr  4 11:46:46 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Fri, 4 Apr 2008 11:46:46 +0200
Subject: [Python-Dev] __eq__ vs hash
Message-ID: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>

Hi all,

the news file for python 2.6 does not mention that you need to define
__hash__ in case you define __eq__ for a class.
This breaks some code (for me: mercurial and pyparsing).
Shouldn't this be documented somewhere (I also cannot find it in the
whatsnew file).

- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080404/a00b2a48/attachment-0001.htm 

From tnelson at onresolve.com  Fri Apr  4 12:57:15 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 4 Apr 2008 03:57:15 -0700
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
 PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln
 PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile
 Tools/buildbot/build-amd64.bat Tools/buildbot/bui...
In-Reply-To: <47F5E5EF.5010005@cheimes.de>
References: <20080403182707.506A41E400D@bag.python.org>,
	<47F5E5EF.5010005@cheimes.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB286@EXMBX04.exchhosting.com>


> I don't like the part where the solution kills the Python process during
> a rebuild. It's too surprising for the user.

Hmmm.  When you say rebuild, I assume you mean the change I made to the pythoncore project's pre-link step to call kill_python.exe, and not to the post-build step of kill_python that runs itself?  Personally, when I'm doing development, if I've got the pcbuild\python_d.exe console open, it's usually to test one liners, I'm not using it to do anything important.  If I forget to close it before I kick off a build, it's annoying running into errors at the link stage, I'd certainly prefer the build system to kill off anything that'll inhibit a successful link before actually linking.

What do others that do Windows development think?  I don't have a problem changing the build behaviour if the approach I've taken is generally disliked.

   Trent.

________________________________________
From: Christian Heimes [lists at cheimes.de]
Sent: 04 April 2008 09:25
To: python-checkins at python.org; Trent Nelson
Subject: Re: r62129 - in python/trunk: PCbuild/debug.vsprops   PCbuild/kill_python.c PCbuild/kill_python.vcproj   PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj   PCbuild/release.vsprops Tools/buildbot/Makefile   Tools/buildbot/build-amd64.bat Tools/buildbot/bui...

trent.nelson schrieb:
> Author: trent.nelson
> Date: Thu Apr  3 20:27:06 2008
> New Revision: 62129
>
> Added:
>    python/trunk/PCbuild/kill_python.c   (contents, props changed)
>    python/trunk/PCbuild/kill_python.vcproj
> Removed:
>    python/trunk/Tools/buildbot/Makefile
>    python/trunk/Tools/buildbot/kill_python.bat
>    python/trunk/Tools/buildbot/kill_python.c
>    python/trunk/Tools/buildbot/kill_python.mak
> Modified:
>    python/trunk/PCbuild/debug.vsprops
>    python/trunk/PCbuild/pcbuild.sln
>    python/trunk/PCbuild/pythoncore.vcproj
>    python/trunk/PCbuild/release.vsprops
>    python/trunk/Tools/buildbot/build-amd64.bat
>    python/trunk/Tools/buildbot/build.bat
>    python/trunk/Tools/buildbot/buildmsi.bat
> Log:
> Reimplement kill_python.  The existing version had a number of flaws, namely, it didn't work for x64 and it wasn't precise about which python_d.exe it was killing -- it just killed the first one it came across that happened to have 'pcbuild\python_d.exe' or 'build\python_d.exe' in it's path.  The new version has been rewritten from the ground up and now lives in PCbuild, instead of Tools\buildbot, and it has also been incorporated into the Visual Studio solution (pcbuild.sln) as 'kill_python'.  The solution has also been altered such that kill_python is called where necessary in the build process in order to prevent any linking errors due to open file locks.  In lieu of this, all of the existing bits and pieces in Tools\buildbot that called out to kill_python at various points have also be
>  en removed as they are now obsolete.  Tested on both Win32 and x64.

I don't like the part where the solution kills the Python process during
a rebuild. It's too surprising for the user.

Christian

From amauryfa at gmail.com  Fri Apr  4 14:55:38 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 4 Apr 2008 14:55:38 +0200
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
	PCbuild/kill_python.c PCbuild/kill_python.vcproj
	PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj
	PCbuild/release.vsprops Tools/buildbot/Makefile
	Tools/buildbot/build-amd64.bat Tools/build
Message-ID: <e27efe130804040555y16f6e8a2y8737d76ceb97f658@mail.gmail.com>

On Fri, Apr 4, 2008 at 12:57 PM, Trent Nelson <tnelson at onresolve.com> wrote:
>
>  > I don't like the part where the solution kills the Python process during
>  > a rebuild. It's too surprising for the user.
>
>  Hmmm.  When you say rebuild, I assume you mean the change I made to the pythoncore project's pre-link step to call kill_python.exe, and not to the post-build step of kill_python that runs itself?  Personally, when I'm doing development, if I've got the pcbuild\python_d.exe console open, it's usually to test one liners, I'm not using it to do anything important.  If I forget to close it before I kick off a build, it's annoying running into errors at the link stage, I'd certainly prefer the build system to kill off anything that'll inhibit a successful link before actually linking.
>
>  What do others that do Windows development think?  I don't have a problem changing the build behaviour if the approach I've taken is generally disliked.

I agree with Christian: in interactive sessions, the F7 key should not
kill my running testsuite... I prefer the linker errors.
Please do this only for buildbot builds!
Or maybe have it controlled by an enviroment variable.

-- 
Amaury Forgeot d'Arc

From amauryfa at gmail.com  Fri Apr  4 14:55:38 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 4 Apr 2008 14:55:38 +0200
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
	PCbuild/kill_python.c PCbuild/kill_python.vcproj
	PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj
	PCbuild/release.vsprops Tools/buildbot/Makefile
	Tools/buildbot/build-amd64.bat Tools/build
Message-ID: <e27efe130804040555y16f6e8a2y8737d76ceb97f658@mail.gmail.com>

On Fri, Apr 4, 2008 at 12:57 PM, Trent Nelson <tnelson at onresolve.com> wrote:
>
>  > I don't like the part where the solution kills the Python process during
>  > a rebuild. It's too surprising for the user.
>
>  Hmmm.  When you say rebuild, I assume you mean the change I made to the pythoncore project's pre-link step to call kill_python.exe, and not to the post-build step of kill_python that runs itself?  Personally, when I'm doing development, if I've got the pcbuild\python_d.exe console open, it's usually to test one liners, I'm not using it to do anything important.  If I forget to close it before I kick off a build, it's annoying running into errors at the link stage, I'd certainly prefer the build system to kill off anything that'll inhibit a successful link before actually linking.
>
>  What do others that do Windows development think?  I don't have a problem changing the build behaviour if the approach I've taken is generally disliked.

I agree with Christian: in interactive sessions, the F7 key should not
kill my running testsuite... I prefer the linker errors.
Please do this only for buildbot builds!
Or maybe have it controlled by an enviroment variable.

-- 
Amaury Forgeot d'Arc

From musiccomposition at gmail.com  Fri Apr  4 15:38:21 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 4 Apr 2008 08:38:21 -0500
Subject: [Python-Dev] Annotations support in python
In-Reply-To: <ee2a432c0804032149i4bd493e5jed27aef793e6b53@mail.gmail.com>
References: <12e5d0d90804030447u67391abby361d0cabcb919018@mail.gmail.com>
	<ee2a432c0804032149i4bd493e5jed27aef793e6b53@mail.gmail.com>
Message-ID: <1afaf6160804040638m73a2ea3fjff046a17c30f66bb@mail.gmail.com>

On Thu, Apr 3, 2008 at 11:49 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:

> The release schedule for 2.6/3.0 is
> http://www.python.org/dev/peps/pep-0361/
> 3.0 will have the feature, 2.6 may or may not.

There is an issue for this: http://bugs.python.org/issue2331

>
>
> n
>
> On Thu, Apr 3, 2008 at 4:47 AM, Heshan Suriyaarachchi
> <heshan.suri at gmail.com> wrote:
> >  Hi,
> >      I need to work with annotations as it is said in [1]. Currently I
> am
> > using python 2.5.1. I would like to know whether the
> > next release of python will support this feature. If the next version
> > support this feature I would like to know when are you
> >  planning to release it. I used the __future__ module but I could not
> get
> > the annotations with it.
> >
> > [1] -  http://www.python.org/dev/peps/pep-3107/
> >
> > Thanx
> > Heshan Suriyaarachchi
> >
> >
> >
> > _______________________________________________
> >  Python-Dev mailing list
> >  Python-Dev at python.org
> >  http://mail.python.org/mailman/listinfo/python-dev
> >  Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
> >
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080404/abaec5a5/attachment.htm 

From gh at ghaering.de  Fri Apr  4 16:26:40 2008
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Fri, 04 Apr 2008 16:26:40 +0200
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
In-Reply-To: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
Message-ID: <47F63AA0.5040106@ghaering.de>

Hello Barry,

Great job with the releases!

Barry Warsaw wrote:
> On behalf of the Python development team and the Python community, I'm  
> happy to announce the second alpha release of Python 2.6, and the  
> fourth alpha release of Python 3.0. [...]

In case you don't know already, the website is not fully updated, yet:

http://www.python.org/download/releases/2.6/

Top of the page says "2.6a1" instead of "2.6a2".

This file is apparently not yet updated for 2.6alpha2.

http://www.python.org/download/releases/2.6/NEWS.txt


For the 3.0 version:

Top of the page still says "3.0a3".

Cheers,

-- Gerhard

From guido at python.org  Fri Apr  4 16:38:04 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 4 Apr 2008 07:38:04 -0700
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
Message-ID: <ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>

On Fri, Apr 4, 2008 at 2:46 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> the news file for python 2.6 does not mention that you need to define
> __hash__ in case you define __eq__ for a class.
> This breaks some code (for me: mercurial and pyparsing).
> Shouldn't this be documented somewhere (I also cannot find it in the
> whatsnew file).

Well, technically this has always been the requirement.

What specific code breaks? Maybe we need to turn this into a warning
in order to be more backwards compatible?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From schmir at gmail.com  Fri Apr  4 16:53:52 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Fri, 4 Apr 2008 16:53:52 +0200
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
Message-ID: <932f8baf0804040753g74cc5200j738dc29b0b5a4c1c@mail.gmail.com>

On Fri, Apr 4, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:

> On Fri, Apr 4, 2008 at 2:46 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> > the news file for python 2.6 does not mention that you need to define
> > __hash__ in case you define __eq__ for a class.
> > This breaks some code (for me: mercurial and pyparsing).
> > Shouldn't this be documented somewhere (I also cannot find it in the
> > whatsnew file).
>
> Well, technically this has always been the requirement.
>

> What specific code breaks? Maybe we need to turn this into a warning
> in order to be more backwards compatible?
>

I don't feel like digging into mercurial or pyparsing code currently. sorry.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080404/44c2b722/attachment.htm 

From guido at python.org  Fri Apr  4 17:05:44 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 4 Apr 2008 08:05:44 -0700
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <932f8baf0804040753g74cc5200j738dc29b0b5a4c1c@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
	<932f8baf0804040753g74cc5200j738dc29b0b5a4c1c@mail.gmail.com>
Message-ID: <ca471dc20804040805q4af29632i453308ba965aee3d@mail.gmail.com>

Understood. Neither do I. :-) But maybe you could get the authors of
that code into this discussion?

On Fri, Apr 4, 2008 at 7:53 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>
>
>
> On Fri, Apr 4, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
> >
> >
> >
> > On Fri, Apr 4, 2008 at 2:46 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> > > the news file for python 2.6 does not mention that you need to define
> > > __hash__ in case you define __eq__ for a class.
> > > This breaks some code (for me: mercurial and pyparsing).
> > > Shouldn't this be documented somewhere (I also cannot find it in the
> > > whatsnew file).
> >
> > Well, technically this has always been the requirement.
> >
> >
> > What specific code breaks? Maybe we need to turn this into a warning
> > in order to be more backwards compatible?
> >
>
> I don't feel like digging into mercurial or pyparsing code currently. sorry.
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jason.orendorff at gmail.com  Fri Apr  4 17:08:26 2008
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Fri, 4 Apr 2008 10:08:26 -0500
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
Message-ID: <bb8868b90804040808r122bbd5p69d6456434ba70d6@mail.gmail.com>

On Fri, Apr 4, 2008 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:
>  What specific code breaks? Maybe we need to turn this into a warning
>  in order to be more backwards compatible?

I looked at Mercurial.

It doesn't use __hash__ at all.  It uses __eq__ in two files, three total uses:
http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/commands.py
http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/context.py

-j

From status at bugs.python.org  Fri Apr  4 18:06:30 2008
From: status at bugs.python.org (Tracker)
Date: Fri,  4 Apr 2008 18:06:30 +0200 (CEST)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20080404160630.91C05780F1@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (03/28/08 - 04/04/08)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1799 open (+34) / 12577 closed (+13) / 14376 total (+47)

Open issues with patches:   536

Average duration of open issues: 716 days.
Median duration of open issues: 1257 days.

Open Issues Breakdown
   open  1774 (+34)
pending    25 ( +0)

Issues Created Or Reopened (47)
_______________________________

Add gettext.pgettext() and variants support                      03/29/08
       http://bugs.python.org/issue2504    created  genepi                    
                                                                               

Easier creation of _ast nodes                                    03/29/08
CLOSED http://bugs.python.org/issue2505    created  georg.brandl              
       patch                                                                   

Line tracing of continue after always-taken if is incorrect      03/29/08
CLOSED http://bugs.python.org/issue2506    created  nedbat                    
                                                                               

Exception state lives too long in 3.0                            03/29/08
       http://bugs.python.org/issue2507    created  pitrou                    
       patch                                                                   

When you create a file using file(path, "w") if the filename is  03/29/08
CLOSED http://bugs.python.org/issue2508    created  kobipe3                   
                                                                               

Bazaar ignore file                                               03/29/08
CLOSED http://bugs.python.org/issue2509    created  benjamin.peterson         
       patch, easy                                                             

Bazaar ignore file                                               03/29/08
       http://bugs.python.org/issue2510    created  benjamin.peterson         
       patch, easy                                                             

Give AST's excepthandler proper attributes                       03/29/08
CLOSED http://bugs.python.org/issue2511    created  georg.brandl              
       patch                                                                   

decide what to do with gettext API                               03/29/08
       http://bugs.python.org/issue2512    created  benjamin.peterson         
       patch                                                                   

64bit cross compilation on windows                               03/30/08
       http://bugs.python.org/issue2513    created  mhammond                  
       patch, 64bit                                                            

new AST init breaks on Store and other fieldless Nodes           03/30/08
CLOSED http://bugs.python.org/issue2514    created  aronacher                 
       patch                                                                   

Segfault while operating on closed sqlite3 cursor.               03/30/08
       http://bugs.python.org/issue2515    created  pjdavis                   
                                                                               

Instance methods are misreporting the number of arguments        03/30/08
       http://bugs.python.org/issue2516    created  belopolsky                
       patch                                                                   

Error when printing an exception containing a Unicode string     03/31/08
       http://bugs.python.org/issue2517    reopened benjamin.peterson         
       patch                                                                   

smtpd.py to handle huge email                                    03/31/08
       http://bugs.python.org/issue2518    created  kawai                     
       patch                                                                   

Typing 'modules' in the interactive help system fails when impor 03/31/08
       http://bugs.python.org/issue2519    created  dennis                    
                                                                               

macerrors.py cannot be imported due to non-ascii characters in c 03/31/08
       http://bugs.python.org/issue2520    created  thecube                   
       patch                                                                   

ABC caches should use weak refs                                  03/31/08
       http://bugs.python.org/issue2521    created  amaury.forgeotdarc        
       26backport                                                              

locale.format() problems with decimal separator                  03/31/08
       http://bugs.python.org/issue2522    created  mishok13                  
       patch                                                                   

binary buffered reading is quadratic                             03/31/08
       http://bugs.python.org/issue2523    created  pitrou                    
                                                                               

IDLE 3.0                                                         03/31/08
CLOSED http://bugs.python.org/issue2524    created  kbk                       
                                                                               

class USTimeZone in Doc/includes/tzinfo-examples.py is out of da 04/01/08
       http://bugs.python.org/issue2525    created  gvanrossum                
       patch, easy                                                             

str.format() :n format does not appear to work                   04/01/08
       http://bugs.python.org/issue2526    created  mark                      
                                                                               

Pass a namespace to timeit                                       04/01/08
       http://bugs.python.org/issue2527    created  potten                    
                                                                               

Change os.access to check ACLs under Windows                     04/01/08
       http://bugs.python.org/issue2528    created  tim.golden                
       patch                                                                   

list/generator comprehension parser doesn't match spec           04/01/08
CLOSED http://bugs.python.org/issue2529    created  kousu                     
                                                                               

Document IO module                                               04/01/08
       http://bugs.python.org/issue2530    created  benjamin.peterson         
                                                                               

float compared to decimal is silently incorrect.                 04/01/08
CLOSED http://bugs.python.org/issue2531    created  jdunck                    
       easy                                                                    

file that breaks 2to3 (despite being correct python)             04/02/08
       http://bugs.python.org/issue2532    created  fijal                     
                                                                               

Invalid TypeError with class method decorator and class method p 04/02/08
CLOSED http://bugs.python.org/issue2533    created  tdimson                   
                                                                               

Speed up isinstance and issubclass                               04/02/08
       http://bugs.python.org/issue2534    created  theller                   
       patch, patch                                                            

duplicate Misc.lower                                             04/02/08
       http://bugs.python.org/issue2535    created  mkiever                   
       patch                                                                   

itertools.permutations docstring is misleading                   04/02/08
       http://bugs.python.org/issue2536    created  ajaksu2                   
                                                                               

re.compile(r'((x|y+)*)*') should fail                            04/02/08
       http://bugs.python.org/issue2537    created  jorendorff                
                                                                               

memoryview of bytes is not readonly                              04/03/08
       http://bugs.python.org/issue2538    created  amaury.forgeotdarc        
       patch                                                                   

Windows Registry issue with 2.5.2 AMD64 msi                      04/03/08
       http://bugs.python.org/issue2539    created  iggyboo                   
                                                                               

If HAVE_LONG_LONG is not defined, longval will not be initialize 04/03/08
       http://bugs.python.org/issue2540    created  ocean-city                
       patch, patch                                                            

Unicode escape sequences not parsed in raw strings.              04/03/08
CLOSED http://bugs.python.org/issue2541    reopened amaury.forgeotdarc        
                                                                               

PyErr_ExceptionMatches must not fail                             04/03/08
       http://bugs.python.org/issue2542    created  theller                   
       patch, patch                                                            

Make ctypes compatible with Python 2.3, 2.4, and 2.5, as per PEP 04/03/08
CLOSED http://bugs.python.org/issue2543    created  theller                   
       patch, patch                                                            

link with "gcc -shared" on HP systems                            04/03/08
CLOSED http://bugs.python.org/issue2544    created  theller                   
       patch, patch                                                            

sphinx.ext.autodoc fails to expand tabs in docstrings            04/04/08
       http://bugs.python.org/issue2545    created  jmillikin                 
                                                                               

Python-2.5.2: crash in visit_decref () at Modules/gcmodule.c:270 04/04/08
       http://bugs.python.org/issue2546    created  garikvm                   
                                                                               

Py30a4 RELNOTES only cover 30a1 and 30a2                         04/04/08
       http://bugs.python.org/issue2547    created  mark                      
                                                                               

Undetected error in exception handling                           04/04/08
       http://bugs.python.org/issue2548    created  theller                   
                                                                               

shutil: NameError (WindowsError) when moving from ext3 to fat32  04/04/08
       http://bugs.python.org/issue2549    created  jerome.chabod             
       easy                                                                    

SO_REUSEADDR doesn't have the same semantics on Windows as on Un 04/04/08
       http://bugs.python.org/issue2550    created  Trent.Nelson              
       patch, 26backport                                                       



Issues Now Closed (45)
______________________

%f format for datetime objects                                    198 days
       http://bugs.python.org/issue1158    georg.brandl              
       patch                                                                   

pythonstartup addition of minor error checking                    136 days
       http://bugs.python.org/issue1442    georg.brandl              
       patch                                                                   

py3k: test_mailbox fails on Windows                               119 days
       http://bugs.python.org/issue1561    amaury.forgeotdarc        
       patch                                                                   

logging documentation is unclear                                  110 days
       http://bugs.python.org/issue1579    vsajip                    
       easy                                                                    

Backport of io.py                                                  65 days
       http://bugs.python.org/issue1919    georg.brandl              
                                                                               

Undocumented lastrowid attribute i sqlite3 cursor class            35 days
       http://bugs.python.org/issue2176    georg.brandl              
       patch                                                                   

test_sqlite fails in 2.6a1 on MacOS X                              28 days
       http://bugs.python.org/issue2215    ghaering                  
                                                                               

TimedRotatingFileHandler does not account for daylight savings t   17 days
       http://bugs.python.org/issue2315    vsajip                    
                                                                               

TimedRotatingFileHandler names files incorrectly if nothing is l   17 days
       http://bugs.python.org/issue2316    vsajip                    
                                                                               

TimedRotatingFileHandler logic for removing files wrong            17 days
       http://bugs.python.org/issue2317    vsajip                    
                                                                               

TimedRotatingFileHandler: rotate every month, or every year        17 days
       http://bugs.python.org/issue2318    vsajip                    
                                                                               

Backport keyword-only arguments to 2.6                             11 days
       http://bugs.python.org/issue2327    georg.brandl              
       26backport                                                              

Logging module hides user code errors (bare except)                14 days
       http://bugs.python.org/issue2424    vsajip                    
                                                                               

pysqlite provides no interface for database SQL dump                9 days
       http://bugs.python.org/issue2426    kippesp                   
       patch                                                                   

Patch for test_urllib2_net moving tests to use a local server       9 days
       http://bugs.python.org/issue2429    gregory.p.smith           
       patch                                                                   

Stop test_format.py from executing as side effect of import         9 days
       http://bugs.python.org/issue2430    georg.brandl              
       patch, patch                                                            

Adding __iter__ to class Values of module optparse                 12 days
       http://bugs.python.org/issue2444    benjamin.peterson         
       patch                                                                   

logging.LogRecord should know how to handle dict-like args          5 days
       http://bugs.python.org/issue2473    vsajip                    
                                                                               

parser support for future import of unicode_strings                 4 days
       http://bugs.python.org/issue2477    nnorwitz                  
       patch                                                                   

Bytearray and io backport                                           4 days
       http://bugs.python.org/issue2479    georg.brandl              
       26backport                                                              

bdb modernized                                                      1 days
       http://bugs.python.org/issue2498    benjamin.peterson         
       patch                                                                   

Sync _sqlite module code                                            1 days
       http://bugs.python.org/issue2500    ghaering                  
                                                                               

Add enum() example for named tuples                                 1 days
       http://bugs.python.org/issue2502    rhettinger                
       patch                                                                   

Replace "== None/True/False" with "is"                              1 days
       http://bugs.python.org/issue2503    benjamin.peterson         
       patch                                                                   

Easier creation of _ast nodes                                       1 days
       http://bugs.python.org/issue2505    georg.brandl              
       patch                                                                   

Line tracing of continue after always-taken if is incorrect         3 days
       http://bugs.python.org/issue2506    nedbat                    
                                                                               

When you create a file using file(path, "w") if the filename is     0 days
       http://bugs.python.org/issue2508    loewis                    
                                                                               

Bazaar ignore file                                                  0 days
       http://bugs.python.org/issue2509    benjamin.peterson         
       patch, easy                                                             

Give AST's excepthandler proper attributes                          1 days
       http://bugs.python.org/issue2511    georg.brandl              
       patch                                                                   

new AST init breaks on Store and other fieldless Nodes              0 days
       http://bugs.python.org/issue2514    georg.brandl              
       patch                                                                   

IDLE 3.0                                                            0 days
       http://bugs.python.org/issue2524    kbk                       
                                                                               

list/generator comprehension parser doesn't match spec              1 days
       http://bugs.python.org/issue2529    kousu                     
                                                                               

float compared to decimal is silently incorrect.                    1 days
       http://bugs.python.org/issue2531    facundobatista            
       easy                                                                    

Invalid TypeError with class method decorator and class method p    0 days
       http://bugs.python.org/issue2533    amaury.forgeotdarc        
                                                                               

Unicode escape sequences not parsed in raw strings.                 0 days
       http://bugs.python.org/issue2541    benjamin.peterson         
                                                                               

Make ctypes compatible with Python 2.3, 2.4, and 2.5, as per PEP    1 days
       http://bugs.python.org/issue2543    theller                   
       patch, patch                                                            

link with "gcc -shared" on HP systems                               1 days
       http://bugs.python.org/issue2544    theller                   
       patch, patch                                                            

file (& socket) I/O are not thread safe                          2052 days
       http://bugs.python.org/issue595601  georg.brandl              
                                                                               

compiler.transformer: correct lineno attribute when possible     1311 days
       http://bugs.python.org/issue1015989 benjamin.peterson         
       patch                                                                   

automatic XMLRPC boxcarring via multicall for xmlrpclib          1250 days
       http://bugs.python.org/issue1054434 loewis                    
       patch                                                                   

Segfaults with concurrent sqlite db access and schema change      582 days
       http://bugs.python.org/issue1546263 ghaering                  
                                                                               

Python is dumping core after the test test_ctypes                 528 days
       http://bugs.python.org/issue1582742 theller                   
                                                                               

sqlite3 documentation omits: close(), commit(), autocommit        454 days
       http://bugs.python.org/issue1625205 ghaering                  
                                                                               

VC6 build patch for trunk                                         353 days
       http://bugs.python.org/issue1700463 ocean-city                
       patch                                                                   

RuntimeWarning: tp_compare didn't return -1 or -2                 300 days
       http://bugs.python.org/issue1733757 amaury.forgeotdarc        
                                                                               



Top Issues Most Discussed (10)
______________________________

 14 Error when printing an exception containing a Unicode string       4 days
open    http://bugs.python.org/issue2517   

 14 Line tracing of continue after always-taken if is incorrect        3 days
closed  http://bugs.python.org/issue2506   

 13 Bazaar ignore file                                                 6 days
open    http://bugs.python.org/issue2510   

 11 64bit cross compilation on windows                                 5 days
open    http://bugs.python.org/issue2513   

 10 Fast BytesIO implementation + misc changes                        88 days
open    http://bugs.python.org/issue1751   

  8 thread unsafe file objects cause crash                          1647 days
open    http://bugs.python.org/issue815646 

  6 Instance methods are misreporting the number of arguments          5 days
open    http://bugs.python.org/issue2516   

  6 run_setup can fail if the setup script uses __file__              17 days
open    http://bugs.python.org/issue2385   

  6 urllib2 header capitalization                                     24 days
open    http://bugs.python.org/issue2275   

  6 implement warnings module in C                                   451 days
pending http://bugs.python.org/issue1631171




From tnelson at onresolve.com  Fri Apr  4 18:07:22 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 4 Apr 2008 09:07:22 -0700
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB280@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB280@EXMBX04.exchhosting.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB28D@EXMBX04.exchhosting.com>

I've raised issue 2550 to track this problem.  I've also provided a patch on the tracker to test_socket.py that reproduces the issue.  Anyone mind if I commit this to trunk?  I'd like to observe if any other platforms exhibit different behaviour via buildbots.  It'll cause all the Windows slaves to fail on test_socket though.  (I can revert it once I've seen how the buildbots behave until I can come up with an actual patch for Windows that fixes the issue.)

http://bugs.python.org/issue2550
http://bugs.python.org/file9939/test_socket.py.patch

    Trent.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [tnelson at onresolve.com]
Sent: 03 April 2008 22:40
To: python-dev at python.org
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between Windows vs Unix (or why test_asynchat is sometimes dying on Windows)

I started looking into this:

http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/289/step-test/0

Pertinent part:

test_asyncore
<snip>
test_asynchat

command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1
remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
]

I tried to replicate it on the buildbot in order to debug, which, surprisingly, I could do consistently by just running rt.bat -q -d -uall test_asynchat.  As the log above indicates, the python process becomes completely and utterly wedged, to the point that I can't even attach a remote debugger and step into it.

Digging further, I noticed that if I ran the following code in two different python consoles, EADDRINUSE was *NOT* being raised by socket.bind():

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('127.0.0.1', 54322))

However, take out the setsockopt line, and wallah, the second s.bind() will raise EADDRINUSE, as expected.  This manifests into a really bizarre issue with test_asynchat in particualr, as subsequent sock.accept() calls on the socket put python into the uber wedged state (can't even ctrl-c out at the console, need to kill the process directly).

Have to leave the office and head home so I don't have any more time to look at it tonight -- just wanted to post here for others to mull over.

    Trent.
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From guido at python.org  Fri Apr  4 18:08:16 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 4 Apr 2008 09:08:16 -0700
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <bb8868b90804040808r122bbd5p69d6456434ba70d6@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
	<bb8868b90804040808r122bbd5p69d6456434ba70d6@mail.gmail.com>
Message-ID: <ca471dc20804040908t2c78876bh6281df9a33334b9d@mail.gmail.com>

On Fri, Apr 4, 2008 at 8:08 AM, Jason Orendorff
<jason.orendorff at gmail.com> wrote:
> On Fri, Apr 4, 2008 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:
>
> >  What specific code breaks? Maybe we need to turn this into a warning
>  >  in order to be more backwards compatible?
>
>  I looked at Mercurial.
>
>  It doesn't use __hash__ at all.  It uses __eq__ in two files, three total uses:
>  http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/commands.py
>  http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/context.py

If it breaks, perhaps the instances of one of those classes that
define __eq__ but not __hash__ are used in a set or as dict keys?
That's the difference between 2.5 and 2.6 in this area.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From p.f.moore at gmail.com  Fri Apr  4 18:30:57 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 4 Apr 2008 17:30:57 +0100
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <ca471dc20804040908t2c78876bh6281df9a33334b9d@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
	<bb8868b90804040808r122bbd5p69d6456434ba70d6@mail.gmail.com>
	<ca471dc20804040908t2c78876bh6281df9a33334b9d@mail.gmail.com>
Message-ID: <79990c6b0804040930y4d29d6a8s1805c4879719c2a@mail.gmail.com>

On 04/04/2008, Guido van Rossum <guido at python.org> wrote:
>  >  It doesn't use __hash__ at all.  It uses __eq__ in two files, three total uses:
>  >  http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/commands.py
>  >  http://hg.intevation.org/mercurial/crew/file/6c4e12682fb9/mercurial/context.py
>
>
> If it breaks, perhaps the instances of one of those classes that
>  define __eq__ but not __hash__ are used in a set or as dict keys?
>  That's the difference between 2.5 and 2.6 in this area.

That looks like it. I'll work up a patch and submit it to the
Mercurial developers.

Paul.

From dirkjan at ochtman.nl  Fri Apr  4 19:29:58 2008
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Fri, 04 Apr 2008 19:29:58 +0200
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <79990c6b0804040930y4d29d6a8s1805c4879719c2a@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>	<bb8868b90804040808r122bbd5p69d6456434ba70d6@mail.gmail.com>	<ca471dc20804040908t2c78876bh6281df9a33334b9d@mail.gmail.com>
	<79990c6b0804040930y4d29d6a8s1805c4879719c2a@mail.gmail.com>
Message-ID: <47F66596.2020507@ochtman.nl>

Paul Moore wrote:
> That looks like it. I'll work up a patch and submit it to the
> Mercurial developers.

I've already got one going.

Cheers,

Dirkjan


From theller at ctypes.org  Fri Apr  4 20:52:24 2008
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 04 Apr 2008 20:52:24 +0200
Subject: [Python-Dev] osx x86 buildbot offer
Message-ID: <ft5td8$vkb$1@ger.gmane.org>

I can offer an OS X x86 machine to run a buildbot on.  This is a physical machine,
running OS X 10.5 Leopard.

Thomas


From tjreedy at udel.edu  Fri Apr  4 20:54:28 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 4 Apr 2008 14:54:28 -0400
Subject: [Python-Dev] fixing tests on windows
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com><d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com><47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk><52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com><47F33DE7.3030702@timgolden.me.uk>	<47F3592D.3080700@timgolden.me.uk><47F4DE23.3020202@timgolden.me.uk>	<ft3h2b$ftl$1@ger.gmane.org>	<d11dcfba0804031452m911e1f3w4ec68a5967d69656@mail.gmail.com><ft3t26$j8a$1@ger.gmane.org>
	<47F5E109.4090608@timgolden.me.uk>
Message-ID: <ft5th0$ii$1@ger.gmane.org>


"Tim Golden" <mail at timgolden.me.uk> wrote in message 
news:47F5E109.4090608 at timgolden.me.uk...
| Terry Reedy wrote:
| > I suspect, but do not know, that the dialog box effects changes through
| > user-programmable interfaces.  So while I would start with manual 
changes
| > to see if that solves the problem, I presume there must be a system 
call
| > for changing dir attributes.
|
| The problem is, I think, that it isn't just the Indexing Service
| which generates this issue. TortoiseSVN is well known for doing
| the same thing, and there could be now and potentially will be
| in the future other programs. I don't think that hunting down
| and turning off their interference case by case is a viable
| solution in the long-term.

And that is exactly not what I was suggesting.  I do not know how much 
privitizing a directory actually blocks, but if Microsoft says 'only you 
can access the folder', perhaps it would block some things.

Don't most scanners read files rapidly, close, and move on?  Perhaps 
testall could either hang around at the end trying to make sure all is 
removed.  Perhaps display a message, wait five seconds, try again, up to a 
minute or so.

tjr




From martin at v.loewis.de  Fri Apr  4 20:57:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 04 Apr 2008 20:57:08 +0200
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
 PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln
 PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile
 Tools/buildbot/build-amd64.bat Tools/buildbot/bui...
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB286@EXMBX04.exchhosting.com>
References: <20080403182707.506A41E400D@bag.python.org>,
	<47F5E5EF.5010005@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB286@EXMBX04.exchhosting.com>
Message-ID: <47F67A04.4020805@v.loewis.de>

> What do others that do Windows development think?  I don't have a
> problem changing the build behaviour if the approach I've taken is
> generally disliked.

I think kill_python should only ever be invoked in the build slaves;
it should *not* be part of the regular build. If developers find they
can't build because some files are still open, they should kill
the processes themselves.

Regards,
Martin


From tjreedy at udel.edu  Fri Apr  4 21:03:14 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 4 Apr 2008 15:03:14 -0400
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
	<47F63AA0.5040106@ghaering.de>
Message-ID: <ft5u1e$2k1$1@ger.gmane.org>


"Gerhard H?ring" <gh at ghaering.de> wrote in message 
news:47F63AA0.5040106 at ghaering.de...
| In case you don't know already, the website is not fully updated, yet:
|
| http://www.python.org/download/releases/2.6/
|
| Top of the page says "2.6a1" instead of "2.6a2".
|
| This file is apparently not yet updated for 2.6alpha2.
|
| http://www.python.org/download/releases/2.6/NEWS.txt
|
|
| For the 3.0 version:
|
| Top of the page still says "3.0a3".

The tops of the pages are fixed, but the left sidebar DOWNLOAD/Releases 
list still says 6a1 and 0a3. 




From tnelson at onresolve.com  Fri Apr  4 21:41:43 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 4 Apr 2008 12:41:43 -0700
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
 PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln
 PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile
 Tools/buildbot/build-amd64.bat Tools/buildbo...
In-Reply-To: <47F67A04.4020805@v.loewis.de>
References: <20080403182707.506A41E400D@bag.python.org>,
	<47F5E5EF.5010005@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB286@EXMBX04.exchhosting.com>,
	<47F67A04.4020805@v.loewis.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB290@EXMBX04.exchhosting.com>

Ok, I'll change the approach this weekend.

    Trent.

________________________________________
From: "Martin v. L?wis" [martin at v.loewis.de]
Sent: 04 April 2008 19:57
To: Trent Nelson
Cc: Christian Heimes; python-dev at python.org
Subject: Re: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile Tools/buildbot/build-amd64.bat Tools/buildbo...

> What do others that do Windows development think?  I don't have a
> problem changing the build behaviour if the approach I've taken is
> generally disliked.

I think kill_python should only ever be invoked in the build slaves;
it should *not* be part of the regular build. If developers find they
can't build because some files are still open, they should kill
the processes themselves.

Regards,
Martin

From barry at python.org  Fri Apr  4 21:54:58 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 4 Apr 2008 15:54:58 -0400
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
In-Reply-To: <ft5u1e$2k1$1@ger.gmane.org>
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
	<47F63AA0.5040106@ghaering.de> <ft5u1e$2k1$1@ger.gmane.org>
Message-ID: <E7B09BA2-A4A6-4D65-AE87-DFE2EB37EBCB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 4, 2008, at 3:03 PM, Terry Reedy wrote:
>
> "Gerhard H?ring" <gh at ghaering.de> wrote in message
> news:47F63AA0.5040106 at ghaering.de...
> | In case you don't know already, the website is not fully updated,  
> yet:
> |
> | http://www.python.org/download/releases/2.6/
> |
> | Top of the page says "2.6a1" instead of "2.6a2".
> |
> | This file is apparently not yet updated for 2.6alpha2.
> |
> | http://www.python.org/download/releases/2.6/NEWS.txt
> |
> |
> | For the 3.0 version:
> |
> | Top of the page still says "3.0a3".
>
> The tops of the pages are fixed, but the left sidebar DOWNLOAD/ 
> Releases
> list still says 6a1 and 0a3.

I think I fixed these -- at least a grep of my local tree turns up no  
other references to 2.6a1 or 3.0a3.  It might take a little while for  
the site to get rebuilt though.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/aHknEjvBPtnXfVAQJ3TQP+IrKnzpxWlJoEDovpg4hb2h4mcVhB5h6+
cpKczLE6DWlC4H4t76hZdRIOf/unAX6FeaTzlA7xLiXNkIlBT455k39jL98ZUmPy
OjOw6n96WyhWL6zmVBnF4YuBFKo2D1Sj9rzYF8i1qtwY3Xq0rvtSSTNmVSnqXDOu
x6H5Z5lm1AY=
=Qzkl
-----END PGP SIGNATURE-----

From gregor.lingl at aon.at  Fri Apr  4 22:07:07 2008
From: gregor.lingl at aon.at (Gregor Lingl)
Date: Fri, 04 Apr 2008 22:07:07 +0200
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
In-Reply-To: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
Message-ID: <47F68A6B.9060901@aon.at>

Hi, something doesn't work as usual, at least for me:

When I try to download the Python 2.6a2 release for Windows by clicking

    * Windows x86 MSI Installer (2.6a2)
      <http://www.python.org/ftp/python/2.6/python-2.6a2.msi> (sig)
      <http://www.python.org/download/releases/2.6/python-2.6a2.msi.asc>

instead of the usual download dialog I get an

Error 404: File not found

for the url

http://www.python.org/ftp/python/2.6/python-2.6a2.msi .

The same is true when trying to download Python 3.0 a4.

Embarassed,
Gregor



Barry Warsaw schrieb:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On behalf of the Python development team and the Python community, I'm  
> happy to announce the second alpha release of Python 2.6, and the  
> fourth alpha release of Python 3.0.
>
> Please note that these are alpha releases, and as such are not  
> suitable for production environments.  We continue to strive for a  
> high degree of quality, but there are still some known problems and  
> the feature sets have not been finalized.  These alphas are being  
> released to solicit feedback and hopefully discover bugs, as well as  
> allowing you to determine how changes in 2.6 and 3.0 might impact  
> you.  If you find things broken or incorrect, please submit a bug  
> report at
>
>     http://bugs.python.org
>
> For more information and downloadable distributions, see the Python  
> 2.6 web
> site:
>
>     http://www.python.org/download/releases/2.6/
>
> and the Python 3.0 web site:
>
>     http://www.python.org/download/releases/3.0/
>
> We are planning one more alpha release of each version, followed by  
> two beta releases, with the final releases planned for August 2008.   
> See PEP 361 for release details:
>
>      http://www.python.org/dev/peps/pep-0361/
>
> Enjoy,
> - -Barry
>
> Barry Warsaw
> barry at python.org
> Python 2.6/3.0 Release Manager
> (on behalf of the entire python-dev team)
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (Darwin)
>
> iQCVAwUBR/WImHEjvBPtnXfVAQJmoQP+MzqNDI+Xt8zua/FE7Ca4TVXoIIy2uoOm
> I1i3+vmevZ9vtAb9hcGwfEgPY4LSwb9Js4KnJJWMPaMuFJK4NgGoiMdj+t42zDbQ
> bEzfBUOCoVkejLRxIQnWeJf1Hu8JocYyCHIRffv57/QdKpHuiSs8aE8GIT3STo3o
> I88H5NY1GgI=
> =WT2z
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/gregor.lingl%40aon.at
>
>
>   

From martin at v.loewis.de  Fri Apr  4 22:23:37 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 04 Apr 2008 22:23:37 +0200
Subject: [Python-Dev] [Python-3000]  RELEASED Python 2.6a2 and 3.0a4
In-Reply-To: <47F68A6B.9060901@aon.at>
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org>
	<47F68A6B.9060901@aon.at>
Message-ID: <47F68E49.30306@v.loewis.de>

> Error 404: File not found

That has a simple explanation: the file is not there because it just
doesn't exist yet, which in turn is because I have problems creating
it (which is in turn due to switching to Visual Studio 2008).

Regards,
Martin

From tnelson at onresolve.com  Fri Apr  4 22:24:49 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 4 Apr 2008 13:24:49 -0700
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB28D@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB280@EXMBX04.exchhosting.com>,
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB28D@EXMBX04.exchhosting.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB291@EXMBX04.exchhosting.com>

Interesting results!  I committed the patch to test_socket.py in r62152.  I was expecting all other platforms except for Windows to behave consistently (i.e. pass).  That is, given the following:

        import socket
        host = '127.0.0.1'
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.bind((host, 0))
        port = sock.getsockname()[1]
        sock.close()
        del sock

        sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        sock1.bind((host, port))
        sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock2.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        sock2.bind((host, port))
        ^^^^

....the second bind should fail with EADDRINUSE, at least according to the 'SO_REUSEADDR and SO_REUSEPORT Socket Options' section in chapter 7.5 of Stevens' UNIX Network Programming Volume 1 (2nd Ed):

"With TCP, we are never able to start multiple servers that bind
 the same IP address and same port: a completely duplicate binding.
 That is, we cannot start one server that binds 198.69.10.2 port 80
 and start another that also binds 198.69.10.2 port 80, even if we
 set the SO_REUSEADDR socket option for the second server."

The results: both Windows *and* Linux fail the patched test; none of the buildbots for either platform encountered an EADDRINUSE socket.error after the second bind().  FreeBSD, OS X, Solaris and Tru64 pass the test -- EADDRINUSE is raised on the second bind.  (Interesting that all the ones that passed have a BSD lineage.)

I've just reverted the test in r62156 as planned.  The real issue now is that there are tests that are calling test_support.bind_socket() with the assumption that the port returned by this method is 'unbound', when in fact, the current implementation can't guarantee this:

def bind_port(sock, host='', preferred_port=54321):
    for port in [preferred_port, 9907, 10243, 32999, 0]:
        try:
            sock.bind((host, port))
            if port == 0:
                port = sock.getsockname()[1]
            return port
        except socket.error, (err, msg):
            if err != errno.EADDRINUSE:
                raise
            print >>sys.__stderr__, \
                '  WARNING: failed to listen on port %d, trying another' % port

This logic is only correct for platforms other than Windows and Linux.  I haven't looked into all the networking test cases that rely on bind_port(), but I would think an implementation such as this would be much more reliable than what we've got for returning an unused port:

def bind_port(sock, host='127.0.0.1', *args):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((host, 0))
    port = s.getsockname()[1]
    s.close()
    del s

    sock.bind((host, port))
    return port

Actually, FWIW, I just ran a full regrtest.py against trunk on Win32 with this change in place and all the tests still pass.

Thoughts?

    Trent.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [tnelson at onresolve.com]
Sent: 04 April 2008 17:07
To: python-dev at python.org
Subject: Re: [Python-Dev] socket.SOL_REUSEADDR: different semantics between Windows vs Unix (or why test_asynchat is sometimes dying on Windows)

I've raised issue 2550 to track this problem.  I've also provided a patch on the tracker to test_socket.py that reproduces the issue.  Anyone mind if I commit this to trunk?  I'd like to observe if any other platforms exhibit different behaviour via buildbots.  It'll cause all the Windows slaves to fail on test_socket though.  (I can revert it once I've seen how the buildbots behave until I can come up with an actual patch for Windows that fixes the issue.)

http://bugs.python.org/issue2550
http://bugs.python.org/file9939/test_socket.py.patch

    Trent.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [tnelson at onresolve.com]
Sent: 03 April 2008 22:40
To: python-dev at python.org
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between Windows vs Unix (or why test_asynchat is sometimes dying on Windows)

I started looking into this:

http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/289/step-test/0

Pertinent part:

test_asyncore
<snip>
test_asynchat

command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1
remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
]

I tried to replicate it on the buildbot in order to debug, which, surprisingly, I could do consistently by just running rt.bat -q -d -uall test_asynchat.  As the log above indicates, the python process becomes completely and utterly wedged, to the point that I can't even attach a remote debugger and step into it.

Digging further, I noticed that if I ran the following code in two different python consoles, EADDRINUSE was *NOT* being raised by socket.bind():

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('127.0.0.1', 54322))

However, take out the setsockopt line, and wallah, the second s.bind() will raise EADDRINUSE, as expected.  This manifests into a really bizarre issue with test_asynchat in particualr, as subsequent sock.accept() calls on the socket put python into the uber wedged state (can't even ctrl-c out at the console, need to kill the process directly).

Have to leave the office and head home so I don't have any more time to look at it tonight -- just wanted to post here for others to mull over.

    Trent.
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From barry at python.org  Fri Apr  4 22:39:09 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 4 Apr 2008 16:39:09 -0400
Subject: [Python-Dev] Python source code on Bazaar vcs
In-Reply-To: <18409.16093.799920.286191@montanaro-dyndns-org.local>
References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org>
	<18408.27695.339064.649345@montanaro-dyndns-org.local>
	<08E8188C-AEA2-4E78-B74C-AF213757DEA0@python.org>
	<18409.16093.799920.286191@montanaro-dyndns-org.local>
Message-ID: <99AE707E-C992-42B9-A238-F2751A691DFA@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2008, at 2:05 PM, skip at pobox.com wrote:
>
>>> Did I misread the directions or do I really need the --create-prefix
>>> arg?
>
>    Barry> You do, the first time you push a user branch because  
> users/skip
>    Barry> doesn't exist yet.  It's mentioned in the docs, but it's  
> pretty
>    Barry> easy to overlook ;).
>
> Well, I noticed the mention in .../dev/bazaar, where it reads, "the  
> first
> time you do this, you might need to add --create-prefix".  Perhaps  
> that
> should read "... you will need to ...".
>
> It pushed fine with --create-prefix.

Thanks Skip.  Fixed.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/aR7XEjvBPtnXfVAQIoXgQAsN6Hfs1JjdFMOI1/Ef2kLeeBfTPxf+Ys
K9y81yEHUNonaQEIF9ptnyOIEyic5uX+Ig4cYO20i1LgvGEIIiCg191EJtYFc9jr
s1dTgmE3PQfiR7J2m2SWS06bYMsanBdAAW/ZnMpgmUMZixYEX43z7Q+kjFibwTn+
UbGz2uLeW+o=
=Fx0S
-----END PGP SIGNATURE-----

From exarkun at divmod.com  Sat Apr  5 00:01:53 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 4 Apr 2008 17:01:53 -0500
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB291@EXMBX04.exchhosting.com>
Message-ID: <20080404220153.6859.1817508967.divmod.quotient.25765@ohm>

On Fri, 4 Apr 2008 13:24:49 -0700, Trent Nelson <tnelson at onresolve.com> wrote:
>Interesting results!  I committed the patch to test_socket.py in r62152.  I was expecting all other platforms except for Windows to behave consistently (i.e. pass).  That is, given the following:
>
>        import socket
>        host = '127.0.0.1'
>        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>        sock.bind((host, 0))
>        port = sock.getsockname()[1]
>        sock.close()
>        del sock
>
>        sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>        sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
>        sock1.bind((host, port))
>        sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>        sock2.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
>        sock2.bind((host, port))
>        ^^^^
>
>....the second bind should fail with EADDRINUSE, at least according to the 'SO_REUSEADDR and SO_REUSEPORT Socket Options' section in chapter 7.5 of Stevens' UNIX Network Programming Volume 1 (2nd Ed):
>
>"With TCP, we are never able to start multiple servers that bind
> the same IP address and same port: a completely duplicate binding.
> That is, we cannot start one server that binds 198.69.10.2 port 80
> and start another that also binds 198.69.10.2 port 80, even if we
> set the SO_REUSEADDR socket option for the second server."
>
>The results: both Windows *and* Linux fail the patched test; none of the buildbots for either platform encountered an EADDRINUSE socket.error after the second bind().  FreeBSD, OS X, Solaris and Tru64 pass the test -- EADDRINUSE is raised on the second bind.  (Interesting that all the ones that passed have a BSD lineage.)

Notice that the quoted text explains that you cannot start multiple servers
that etc.  Since you didn't call listen on either socket, it's arguable that
you didn't start any servers, so there should be no surprise regarding the
behavior.  Try adding listen calls at various places in the example and
you'll see something different happen.

FWIW, AIUI, SO_REUSEADDR behaves just as described in the above quote on
Linux/BSD/UNIX/etc.  On Windows, however, that option actually means
something quite different.  It means that the address should be stolen from
any process which happens to be using it at the moment.

There is another option, SO_EXCLUSIVEADDRUSE, only on Windows I think,
which, AIUI, makes it impossible for another process to steal the port
using SO_REUSEADDR.

Hope this helps,

Jean-Paul

From martin at martinthomas.net  Sat Apr  5 00:58:53 2008
From: martin at martinthomas.net (Martin Thomas)
Date: Fri, 4 Apr 2008 17:58:53 -0500
Subject: [Python-Dev] Dates wrong on front page of pydotorg
Message-ID: <FAF949BD-6539-44C5-9CDE-22B72549C26D@martinthomas.net>

Did anyone else notice that the dates are incorrect in the news items  
on the front page? As an example:
Published: Mon, 4 Apr 2008
which should be
Published: Fri, 4 Apr 2008

I'll poke around and see if I can figure it out.

Cheers //M





From ggpolo at gmail.com  Sat Apr  5 01:37:14 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 4 Apr 2008 20:37:14 -0300
Subject: [Python-Dev] Dates wrong on front page of pydotorg
In-Reply-To: <FAF949BD-6539-44C5-9CDE-22B72549C26D@martinthomas.net>
References: <FAF949BD-6539-44C5-9CDE-22B72549C26D@martinthomas.net>
Message-ID: <ac2200130804041637g3d9c1c90o32d39ae9fb0b83ab@mail.gmail.com>

2008/4/4, Martin Thomas <martin at martinthomas.net>:
> Did anyone else notice that the dates are incorrect in the news items
>  on the front page? As an example:
>  Published: Mon, 4 Apr 2008
>  which should be
>  Published: Fri, 4 Apr 2008
>
>  I'll poke around and see if I can figure it out.
>

There is a file (newsindex.yml) which contains the news items. The
publication date is hand written so it is just a typo from whoever
commited that news item with that publication date.

I just fixed it (as you pointed out). Thanks ;)

Anyway, did you send this to the wrong list or was it intentional ?

Regards,

>  Cheers //M
>
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>


-- 
-- Guilherme H. Polo Goncalves

From exarkun at divmod.com  Sat Apr  5 01:37:23 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 4 Apr 2008 18:37:23 -0500
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
Message-ID: <20080404233723.6859.1733915494.divmod.quotient.25786@ohm>

On Fri, 4 Apr 2008 07:38:04 -0700, Guido van Rossum <guido at python.org> wrote:
>On Fri, Apr 4, 2008 at 2:46 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>> the news file for python 2.6 does not mention that you need to define
>> __hash__ in case you define __eq__ for a class.
>> This breaks some code (for me: mercurial and pyparsing).
>> Shouldn't this be documented somewhere (I also cannot find it in the
>> whatsnew file).
>
>Well, technically this has always been the requirement.
>
>What specific code breaks? Maybe we need to turn this into a warning
>in order to be more backwards compatible?
>

There was some code in Twisted (one class, specifically) which was broken/
revealed to be broken by this Python 2.6 change.  The code assumed identity
hashing if no __hash__ method was implemented.  This ended up only working
if you only had a singleton instance of the class, but the class also went
out of its way to make sure that was the case.  We have since changed the
code to work on Python 2.6.

If you're curious about the details, here's the code after the fix:

http://twistedmatrix.com/trac/browser/trunk/twisted/web2/dav/element/base.py?rev=22305#L345

Here's the changeset that fixed it:

http://twistedmatrix.com/trac/changeset/22305

And here's the same class before the fix:

http://twistedmatrix.com/trac/browser/trunk/twisted/web2/dav/element/base.py?rev=22304#L344

Jean-Paul

From martin at martinthomas.net  Sat Apr  5 01:47:11 2008
From: martin at martinthomas.net (Martin Thomas)
Date: Fri, 4 Apr 2008 18:47:11 -0500
Subject: [Python-Dev] Dates wrong on front page of pydotorg
In-Reply-To: <ac2200130804041637g3d9c1c90o32d39ae9fb0b83ab@mail.gmail.com>
References: <FAF949BD-6539-44C5-9CDE-22B72549C26D@martinthomas.net>
	<ac2200130804041637g3d9c1c90o32d39ae9fb0b83ab@mail.gmail.com>
Message-ID: <F40AE454-9E9D-42CC-8B5A-D87C0F7C7605@martinthomas.net>


On Apr 4, 2008, at 6:37 PM, Guilherme Polo wrote:

> 2008/4/4, Martin Thomas <martin at martinthomas.net>:
>> Did anyone else notice that the dates are incorrect in the news items
>> on the front page? As an example:
>> Published: Mon, 4 Apr 2008
>> which should be
>> Published: Fri, 4 Apr 2008
>>
>> I'll poke around and see if I can figure it out.
>>
>
> There is a file (newsindex.yml) which contains the news items. The
> publication date is hand written so it is just a typo from whoever
> commited that news item with that publication date.
>
> I just fixed it (as you pointed out). Thanks ;)
Welcome :-)
>
>
> Anyway, did you send this to the wrong list or was it intentional ?
I sent it to the wrong list.. should have pydotorg.  Sorry.
//M


From tjreedy at udel.edu  Sat Apr  5 03:11:09 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 4 Apr 2008 21:11:09 -0400
Subject: [Python-Dev] RELEASED Python 2.6a2 and 3.0a4
References: <3C3C0150-ED65-4381-9F54-BB437DD6DFB9@python.org><47F63AA0.5040106@ghaering.de>
	<ft5u1e$2k1$1@ger.gmane.org>
	<E7B09BA2-A4A6-4D65-AE87-DFE2EB37EBCB@python.org>
Message-ID: <ft6jjc$4mt$1@ger.gmane.org>


"Barry Warsaw" <barry at python.org> wrote in message 
news:E7B09BA2-A4A6-4D65-AE87-DFE2EB37EBCB at python.org...
I think I fixed these -- at least a grep of my local tree turns up no
other references to 2.6a1 or 3.0a3.  It might take a little while for
the site to get rebuilt though.
=============

Fixed now.
Now  I am only waiting for the Windows binaries to be built and added 




From skip at pobox.com  Sat Apr  5 02:37:47 2008
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 4 Apr 2008 19:37:47 -0500
Subject: [Python-Dev] inittimezone - shouldn't it be static?
Message-ID: <18422.51675.824719.583407@montanaro-dyndns-org.local>


Looks to me like the inittimezone function in timemodule.c should be static.
I don't see it called from any code other than within that module.  If it's
not supposed to be static it needs a more pythonic name.  (Can't dig into
this just now as I'm flying from Denver to Chicago a.t.m...)

Skip


From nnorwitz at gmail.com  Sat Apr  5 05:14:04 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 4 Apr 2008 20:14:04 -0700
Subject: [Python-Dev] osx x86 buildbot offer
In-Reply-To: <ft5td8$vkb$1@ger.gmane.org>
References: <ft5td8$vkb$1@ger.gmane.org>
Message-ID: <ee2a432c0804042014t5b61f952u5c7a2a8a0cf46c71@mail.gmail.com>

On Fri, Apr 4, 2008 at 11:52 AM, Thomas Heller <theller at ctypes.org> wrote:
> I can offer an OS X x86 machine to run a buildbot on.  This is a physical machine,
>  running OS X 10.5 Leopard.

Thanks Thomas!

Martin and I will coordinate with you off-list.

n

From antonyjoseph89 at gmail.com  Sat Apr  5 07:49:21 2008
From: antonyjoseph89 at gmail.com (Antony Joseph)
Date: Fri, 4 Apr 2008 22:49:21 -0700
Subject: [Python-Dev] Antony Joseph wants to chat
Message-ID: <f14c4c340804042249i2905ec01v@mail.gmail.com>

-----------------------------------------------------------------------

Antony Joseph wants to stay in better touch using some of Google's coolest new
products.

If you already have Gmail or Google Talk, visit:
http://mail.google.com/mail/b-29833b0a8-b7e4c018a7-1e5ec176e7509c87
You'll need to click this link to be able to chat with Antony Joseph.

To get Gmail - a free email account from Google with over 2,800 megabytes of
storage - and chat with Antony Joseph, visit:
http://mail.google.com/mail/a-29833b0a8-b7e4c018a7-e7105acb3e

Gmail offers:
- Instant messaging right inside Gmail
- Powerful spam protection
- Built-in search for finding your messages and a helpful way of organizing
  emails into "conversations"
- No pop-up ads or untargeted banners - just text ads and related information
  that are relevant to the content of your messages

All this, and its yours for free. But wait, there's more! By opening a Gmail
account, you also get access to Google Talk, Google's instant messaging
service:

http://www.google.com/talk/

Google Talk offers:
- Web-based chat that you can use anywhere, without a download
- A contact list that's synchronized with your Gmail account
- Free, high quality PC-to-PC voice calls when you download the Google Talk
  client

Gmail and Google Talk are still in beta. We're working hard to add new features
and make improvements, so we might also ask for your comments and suggestions
periodically. We appreciate your help in making our products even better!

Thanks,
The Google Team

To learn more about Gmail and Google Talk, visit:
http://mail.google.com/mail/help/about.html
http://www.google.com/talk/about.html

(If clicking the URLs in this message does not work, copy and paste them into
the address bar of your browser).

From guido at python.org  Sat Apr  5 16:46:58 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 5 Apr 2008 07:46:58 -0700
Subject: [Python-Dev] inittimezone - shouldn't it be static?
In-Reply-To: <18422.51675.824719.583407@montanaro-dyndns-org.local>
References: <18422.51675.824719.583407@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804050746l4d879eeepe58d696f6a000d84@mail.gmail.com>

Sounds like an oversight. Please fix!

On Fri, Apr 4, 2008 at 5:37 PM,  <skip at pobox.com> wrote:
>
>  Looks to me like the inittimezone function in timemodule.c should be static.
>  I don't see it called from any code other than within that module.  If it's
>  not supposed to be static it needs a more pythonic name.  (Can't dig into
>  this just now as I'm flying from Denver to Chicago a.t.m...)


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eikeon at eikeon.com  Sat Apr  5 18:48:21 2008
From: eikeon at eikeon.com (Daniel Krech)
Date: Sat, 5 Apr 2008 12:48:21 -0400
Subject: [Python-Dev] __eq__ vs hash
In-Reply-To: <ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
References: <932f8baf0804040246s261c1cd2ic977934cbd049731@mail.gmail.com>
	<ca471dc20804040738h11037a88i5b4022dd2f101008@mail.gmail.com>
Message-ID: <DBDC5179-7394-4F6D-BA76-34364CE396E8@eikeon.com>


On Apr 4, 2008, at 10:38 AM, Guido van Rossum wrote:
> On Fri, Apr 4, 2008 at 2:46 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>> the news file for python 2.6 does not mention that you need to define
>> __hash__ in case you define __eq__ for a class.
>> This breaks some code (for me: mercurial and pyparsing).
>> Shouldn't this be documented somewhere (I also cannot find it in the
>> whatsnew file).
>
> Well, technically this has always been the requirement.
>
> What specific code breaks? Maybe we need to turn this into a warning
> in order to be more backwards compatible?

I think a warning would be nice. I've run into a number of places that  
are breaking due to the change. Including parts of the standard lib,  
in particular, unittest.TestSuite as used by nose [1].

There is already an open issue with patches for this in the issue  
tracker [2]. I've been running trunk with the patch installed and am  
able to get past the breakage due to the change. I don't believe the  
current patch is kicking out a warning however.

[1] http://code.google.com/p/python-nose/issues/detail?id=161
[2] http://bugs.python.org/issue2235


From tnelson at onresolve.com  Sat Apr  5 19:22:44 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Sat, 5 Apr 2008 10:22:44 -0700
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <20080404220153.6859.1817508967.divmod.quotient.25765@ohm>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB291@EXMBX04.exchhosting.com>
	<20080404220153.6859.1817508967.divmod.quotient.25765@ohm>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E226D8C7C@EXMBX04.exchhosting.com>

> >"With TCP, we are never able to start multiple servers that bind
> > the same IP address and same port: a completely duplicate binding.
> > That is, we cannot start one server that binds 198.69.10.2 port 80
> > and start another that also binds 198.69.10.2 port 80, even if we
> > set the SO_REUSEADDR socket option for the second server."

> Notice that the quoted text explains that you cannot start multiple
> servers that etc.  Since you didn't call listen on either socket, it's
> arguable that you didn't start any servers, so there should be no
> surprise regarding the behavior.  Try adding listen calls at various
> places in the example and you'll see something different happen.

I agree in principle, Stevens says nothing about what happens if you *do* try and bind two sockets on two identical host/port addresses.  Even so, test_support.bind_port() makes an assumption that bind() will raise EADDRINUSE if the port is not available, which, as has been demonstrated, won't be the case on Windows or Linux.

> FWIW, AIUI, SO_REUSEADDR behaves just as described in the above quote
> on Linux/BSD/UNIX/etc.  On Windows, however, that option actually means
> something quite different.  It means that the address should be stolen
> from any process which happens to be using it at the moment.

Probably explains why the python process wedges when this happens on Windows...

> There is another option, SO_EXCLUSIVEADDRUSE, only on Windows I think,
> which, AIUI, makes it impossible for another process to steal the port
> using SO_REUSEADDR.

Nod, if SO_EXCLUSIVEADDRUSE is used instead in the code I posted, Windows raises EADDRINUSE on the second bind().  I don't have access to any Linux boxes at the moment, so I can't test what sort of error is raised with the example I posted if listen() and accept() are called on the two sockets bound to identical addresses.  Can anyone else shed some light on this?  I'd be interested in knowing if the process wedges on Linux as badly as it does on Windows (to the point where it's not respecting ctrl-c or sigkill).


        Trent.

From skip at pobox.com  Sat Apr  5 21:17:19 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 5 Apr 2008 14:17:19 -0500
Subject: [Python-Dev] configure error: "rm: conftest.dSYM: is a directory"
Message-ID: <18423.53311.150471.667891@montanaro-dyndns-org.local>

I just noticed this error message during configure:

    checking whether gcc accepts -Olimit 1500... no
    checking whether gcc supports ParseTuple __format__... no
    checking whether pthreads are available without options... yes
    checking whether g++ also accepts flags for thread support... no
    checking for ANSI C header files... rm: conftest.dSYM: is a directory
    rm: conftest.dSYM: is a directory
    yes
    checking for sys/types.h... yes
    checking for sys/stat.h... yes
    checking for stdlib.h... yes
    checking for string.h... yes

Note the "rm: conftest.dSYM: is a directory".  This occurred a few times
during the configure process.  Didn't cause it to conk out, but is annoying.

Skip

From skip at pobox.com  Sat Apr  5 21:49:08 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 5 Apr 2008 14:49:08 -0500
Subject: [Python-Dev] inittimezone - shouldn't it be static?
In-Reply-To: <ca471dc20804050746l4d879eeepe58d696f6a000d84@mail.gmail.com>
References: <18422.51675.824719.583407@montanaro-dyndns-org.local>
	<ca471dc20804050746l4d879eeepe58d696f6a000d84@mail.gmail.com>
Message-ID: <18423.55220.469345.843015@montanaro-dyndns-org.local>

    >> Looks to me like the inittimezone function in timemodule.c should be
    >> static.

    Guido> Sounds like an oversight. Please fix!

Done.  inittimezone is now declared static.

Skip

From mhammond at skippinet.com.au  Sun Apr  6 04:34:54 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Sun, 6 Apr 2008 12:34:54 +1000
Subject: [Python-Dev] Remove "current" Windows executables from
	Lib/distutils/command in svn?
Message-ID: <018701c8978e$ce620660$6b261320$@com.au>

As I was preparing to check-in my cross-compilation patch, which includes a
new x64 executable in the Lib/distutils/command directory, it occurs to me
that we don't actually need it in subversion - and nor do we need
wininst-9.0.exe.

I believe these executables are in svn for historical reasons.  The build
process for these executables are now integrated into the core Python build
process (although they are disabled by default).  There is a good use-case
for keeping executables built with older MSVC versions, but I don't see one
for executables built with the current version.

I'd like to propose we delete Lib/Distutils/command/wininst-9.0.exe, and
enable the building of that project by default in the standard build process
(and I'll setup the x64 build of the executable similarly).  They will then
need to be re-added when a future version moves away from Visual Studio
2008, but that seems like a reasonable way to approach things.  Are there
any objections to this, or should we stick with the status-quo and I'll add
the new x64 executable?  Or do things that way *just* for x64?
 
Thanks,

Mark. 


From martin at v.loewis.de  Sun Apr  6 08:15:32 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Apr 2008 08:15:32 +0200
Subject: [Python-Dev] Remove "current" Windows executables
 from	Lib/distutils/command in svn?
In-Reply-To: <018701c8978e$ce620660$6b261320$@com.au>
References: <018701c8978e$ce620660$6b261320$@com.au>
Message-ID: <47F86A84.6000204@v.loewis.de>

> I'd like to propose we delete Lib/Distutils/command/wininst-9.0.exe, and
> enable the building of that project by default in the standard build process
> (and I'll setup the x64 build of the executable similarly).

There are two issues here:
a) how does the binary get into the release tarball? You might argue
   that it doesn't have to, as it is sufficient when it is included in
   the Windows installer, however, as currently implemented,
   bdist_wininst also runs on Unix, and people use it that way.
b) IIRC, upx was used to compress these executables. I don't think the
   the current build process covers this, yet (but then, the current
   binaries might not be compressed with upx anymore, either - not
   sure whether that would be a bug). Of course, upx would probably
   not currently apply to Win64-amd64.

As another alternative to providing a separate AMD64 binary, you could
also try to make the existing binary work properly on Win64, and tell
it with a config flag whether to look for Win32 or Win64 (or either)
on the target system.

Regards,
Martin

From greg at krypto.org  Sun Apr  6 08:29:29 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 5 Apr 2008 23:29:29 -0700
Subject: [Python-Dev] Thread-safe file objects, the return
In-Reply-To: <ca471dc20804021147h77d89972wb5d1726514433637@mail.gmail.com>
References: <loom.20080331T194117-885@post.gmane.org>
	<ca471dc20804011709x142e57ebj7569529a88b500e4@mail.gmail.com>
	<loom.20080402T100849-143@post.gmane.org>
	<ca471dc20804021147h77d89972wb5d1726514433637@mail.gmail.com>
Message-ID: <52dc1c820804052329i10fe9c75ka6572e37d603ad70@mail.gmail.com>

I've reviewed the patch on http://bugs.python.org/issue815646 and have
uploaded my modified version (mostly test improvements and some formatting
to keep C code under 80 columns with proper 8 space tabs).  I would have
committed it already but I have a sneaking suspicion that its unit test will
barf on windows since it could depend on some posix-like file system
semantics.

Could someone with a windows build environment please test it as asked in
the issue and report back in the tracker?

thanks!
-gps

On Wed, Apr 2, 2008 at 11:47 AM, Guido van Rossum <guido at python.org> wrote:

> On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >  Guido van Rossum <guido <at> python.org> writes:
> >  > Your solution (a counter) seems fine except I think perhaps the
> >  > close() call should not raise IOError -- instead, it should set a
> flag
> >  > so that the thread that makes the counter go to zero can close the
> >  > thread (after all the file got closed while it was being used).
> >
> >  I agree with Gregory: we should be explicit about what happens. I
> wonder
> >  what we would gain from that approach - apart from encouraging
> dangerous
> >  coding practices :)
> >  It also depends how far we want to go: I am merely proposing to fix the
> >  crashes, do we want to provide a "smarter" close() variation that does
> what
> >  you suggest for people that want (or need) to take the risk?
>
> I also agree with Gregory now -- at least on the issue of fclose().
>
> I think that for other (non-closing) operations we should be fine,
> given the Posix requirement that streams have an internal lock. While
> multiple threads reading from a file sounds insane, multiple files
> writing to a file is pretty common (think of stdout or stderr) and
> should be supported.
>
> >  > There are of course other concurrency issues besides close -- what if
> >  > two threads both try to do I/O on the file? What will the C stdio
> >  > library do in that case? Are stdio files thread-safe at the C level?
> >
> >  According to the glibc documentation, at
> >
> http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html:
> >
> >  ? The POSIX standard requires that by default the stream operations are
> >  atomic. I.e., issuing two stream operations for the same stream in two
> >  threads at the same time will cause the operations to be executed as if
> >  they were issued sequentially. The buffer operations performed while
> >  reading or writing are protected from other uses of the same stream. To
> do
> >  this each stream has an internal lock object which has to be
> (implicitly)
> >  acquired before any work can be done. ?
> >
> >  So according to POSIX rules it should be perfectly safe.
>
> Cool.
>
> >  In any case, someone would have to try my patch under Windows and OS X
> and
> >  see if test_file.py passes without crashing.
>
> I know Windows has internal locks on stdio. I trust that OSX, being a
> BSD descendant, is posix compliant. So I'm not worried about these.
>
> +1 on your patch, assuming some other developer reviews it and submits it.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080405/0427aa16/attachment-0001.htm 

From brett at python.org  Sun Apr  6 10:13:57 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 6 Apr 2008 10:13:57 +0200
Subject: [Python-Dev] configure error: "rm: conftest.dSYM: is a
	directory"
In-Reply-To: <18423.53311.150471.667891@montanaro-dyndns-org.local>
References: <18423.53311.150471.667891@montanaro-dyndns-org.local>
Message-ID: <bbaeab100804060113w12ddcef7v81a23a97318b3d78@mail.gmail.com>

On Sat, Apr 5, 2008 at 9:17 PM,  <skip at pobox.com> wrote:
> I just noticed this error message during configure:
>
>     checking whether gcc accepts -Olimit 1500... no
>     checking whether gcc supports ParseTuple __format__... no
>     checking whether pthreads are available without options... yes
>     checking whether g++ also accepts flags for thread support... no
>     checking for ANSI C header files... rm: conftest.dSYM: is a directory
>     rm: conftest.dSYM: is a directory
>     yes
>     checking for sys/types.h... yes
>     checking for sys/stat.h... yes
>     checking for stdlib.h... yes
>     checking for string.h... yes
>
>  Note the "rm: conftest.dSYM: is a directory".  This occurred a few times
>  during the configure process.  Didn't cause it to conk out, but is annoying.

I am assuming this is on your OS X machine, Skip? I have been getting
that message for a while on my OS X machine as well. Same with some
linking message when building ctypes. Never bothered to debug them
since they never seemed to affect anything.

-Brett

From mhammond at skippinet.com.au  Sun Apr  6 15:19:21 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Sun, 6 Apr 2008 23:19:21 +1000
Subject: [Python-Dev] Remove "current" Windows executables
	from	Lib/distutils/command in svn?
In-Reply-To: <47F86A84.6000204@v.loewis.de>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
Message-ID: <01c701c897e8$d8ce3a60$8a6aaf20$@com.au>

> As another alternative to providing a separate AMD64 binary, you could
> also try to make the existing binary work properly on Win64, and tell
> it with a config flag whether to look for Win32 or Win64 (or either)
> on the target system.

I'm afraid that isn't an option for me in the short term.  Should I just
check-in my cross-compilation patch as it stands then?
 
Mark


From skip at pobox.com  Sun Apr  6 15:20:47 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 6 Apr 2008 08:20:47 -0500
Subject: [Python-Dev] configure error: "rm: conftest.dSYM: is a
	directory"
In-Reply-To: <bbaeab100804060113w12ddcef7v81a23a97318b3d78@mail.gmail.com>
References: <18423.53311.150471.667891@montanaro-dyndns-org.local>
	<bbaeab100804060113w12ddcef7v81a23a97318b3d78@mail.gmail.com>
Message-ID: <18424.52783.238721.876293@montanaro-dyndns-org.local>


    >> Note the "rm: conftest.dSYM: is a directory".  This occurred a few
    >> times during the configure process.  Didn't cause it to conk out, but
    >> is annoying.

    Brett> I am assuming this is on your OS X machine, Skip?

Yes, sorry.  I forgot to mention that.  As long as someone else sees it and
feels fine to ignore it, I'm fine with it.  I suspect it's a configure
problem anyway, not a Python problem.

Skip

From martin at v.loewis.de  Sun Apr  6 15:54:48 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 06 Apr 2008 15:54:48 +0200
Subject: [Python-Dev] Remove "current" Windows executables
 from	Lib/distutils/command in svn?
In-Reply-To: <01c701c897e8$d8ce3a60$8a6aaf20$@com.au>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
	<01c701c897e8$d8ce3a60$8a6aaf20$@com.au>
Message-ID: <47F8D628.1030404@v.loewis.de>

>> As another alternative to providing a separate AMD64 binary, you could
>> also try to make the existing binary work properly on Win64, and tell
>> it with a config flag whether to look for Win32 or Win64 (or either)
>> on the target system.
> 
> I'm afraid that isn't an option for me in the short term.  Should I just
> check-in my cross-compilation patch as it stands then?

Sure, go ahead.

Regards,
Martin


From tnelson at onresolve.com  Sun Apr  6 22:51:28 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Sun, 6 Apr 2008 13:51:28 -0700
Subject: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops
 PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln
 PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile
 Tools/buildbot/build-amd64.bat Tools/buildbo...
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB290@EXMBX04.exchhosting.com>
References: <20080403182707.506A41E400D@bag.python.org>,
	<47F5E5EF.5010005@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB286@EXMBX04.exchhosting.com>,
	<47F67A04.4020805@v.loewis.de>,
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB290@EXMBX04.exchhosting.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EE1@EXMBX04.exchhosting.com>

Fixed in r62193.

________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [tnelson at onresolve.com]
Sent: 04 April 2008 20:41
To: "Martin v. L?wis"
Cc: Christian Heimes; python-dev at python.org
Subject: Re: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile Tools/buildbot/build-amd64.bat Tools/buildbo...

Ok, I'll change the approach this weekend.

    Trent.

________________________________________
From: "Martin v. L?wis" [martin at v.loewis.de]
Sent: 04 April 2008 19:57
To: Trent Nelson
Cc: Christian Heimes; python-dev at python.org
Subject: Re: [Python-Dev] r62129 - in python/trunk: PCbuild/debug.vsprops PCbuild/kill_python.c PCbuild/kill_python.vcproj PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj PCbuild/release.vsprops Tools/buildbot/Makefile Tools/buildbot/build-amd64.bat Tools/buildbo...

> What do others that do Windows development think?  I don't have a
> problem changing the build behaviour if the approach I've taken is
> generally disliked.

I think kill_python should only ever be invoked in the build slaves;
it should *not* be part of the regular build. If developers find they
can't build because some files are still open, they should kill
the processes themselves.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From tnelson at onresolve.com  Sun Apr  6 23:34:20 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Sun, 6 Apr 2008 14:34:20 -0700
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
 Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E226D8C7C@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB291@EXMBX04.exchhosting.com>
	<20080404220153.6859.1817508967.divmod.quotient.25765@ohm>,
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E226D8C7C@EXMBX04.exchhosting.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EE2@EXMBX04.exchhosting.com>

I've attached a patch (http://bugs.python.org/file9966/trunk.2550.patch) to issue 2550 that addresses the original problem here: test_support.bind_port() potentially returning ports that have already been bound to.  The patch updates the tests that relied on this method, such that they call it with the new calling convention (test_ftplib, test_httplib, test_socket, test_ssl_socket, test_asynchat, test_telnetlib).

Any objections to the patch?  Would like to commit it sooner rather than later, as it'll fix my buildbots from wedging on test_asynchat at the very least.

    Trent.
________________________________________
From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [tnelson at onresolve.com]
Sent: 05 April 2008 18:22
To: Jean-Paul Calderone; python-dev at python.org
Subject: Re: [Python-Dev] socket.SOL_REUSEADDR: different semantics between Windows vs Unix (or why test_asynchat is sometimes dying on Windows)

> >"With TCP, we are never able to start multiple servers that bind
> > the same IP address and same port: a completely duplicate binding.
> > That is, we cannot start one server that binds 198.69.10.2 port 80
> > and start another that also binds 198.69.10.2 port 80, even if we
> > set the SO_REUSEADDR socket option for the second server."

> Notice that the quoted text explains that you cannot start multiple
> servers that etc.  Since you didn't call listen on either socket, it's
> arguable that you didn't start any servers, so there should be no
> surprise regarding the behavior.  Try adding listen calls at various
> places in the example and you'll see something different happen.

I agree in principle, Stevens says nothing about what happens if you *do* try and bind two sockets on two identical host/port addresses.  Even so, test_support.bind_port() makes an assumption that bind() will raise EADDRINUSE if the port is not available, which, as has been demonstrated, won't be the case on Windows or Linux.

> FWIW, AIUI, SO_REUSEADDR behaves just as described in the above quote
> on Linux/BSD/UNIX/etc.  On Windows, however, that option actually means
> something quite different.  It means that the address should be stolen
> from any process which happens to be using it at the moment.

Probably explains why the python process wedges when this happens on Windows...

> There is another option, SO_EXCLUSIVEADDRUSE, only on Windows I think,
> which, AIUI, makes it impossible for another process to steal the port
> using SO_REUSEADDR.

Nod, if SO_EXCLUSIVEADDRUSE is used instead in the code I posted, Windows raises EADDRINUSE on the second bind().  I don't have access to any Linux boxes at the moment, so I can't test what sort of error is raised with the example I posted if listen() and accept() are called on the two sockets bound to identical addresses.  Can anyone else shed some light on this?  I'd be interested in knowing if the process wedges on Linux as badly as it does on Windows (to the point where it's not respecting ctrl-c or sigkill).


        Trent.
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com

From garg1 at ualberta.ca  Mon Apr  7 02:48:50 2008
From: garg1 at ualberta.ca (Rahul Garg)
Date: Sun, 06 Apr 2008 18:48:50 -0600
Subject: [Python-Dev] New project : Spyke python-to-C compiler
Message-ID: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>

Note this message has been posted to numpy-discussion and python-dev.
Sorry for the multiple posting but I thought both python devs and
numpy users will be interested. If you believe your list should not  
receive this email, let me know. Also I just wanted to introduce  
myself since I may ask doubts about Python and Numpy internals from  
time to time :)

Hi.

I am a student at Univ of Alberta doing my masters in computing science.
I am writing a Python-to-C compiler as one part of my thesis.
The compiler, named Spyke, will be made available in a couple of weeks
and is geared towards scientific applications and will therefore focus  
mostly on needs of scientific app developers.

What is Spyke?
In many performance critical projects, it is often necessary to
rewrite parts of the application in C. However writing C wrappers can
be time consuming. Spyke offers an alternative approach. You add  
annotations to your  Python code as strings. These strings are  
discarded by the Python
interpreter but these are interpreted as types by Spyke compiler to
convert to C.

Example :

"int -> int"
def f(x): return 2*x

In this case the Spyke compiler will consider the string "int -> int"
as a decalration that the function accepts int as parameter and
returns int. Spyke will then generate a C function and a wrapper  
function. This
idea is directly copied from PLW (Python Language Wrapper) project.
Once Python3k arrives, much of these declarations will be moved to
function annotations and class decorators.

This way you can do all your development and debugging interactively
using the standard Python interpreter. When you need to compile to C,
you just add type annotations to places that you want to convert and
invoke spyke on the annotated module. This is different from Pyrex
because Pyrex does not accept Python code. With Spyke, your code is
100% pure python.

Spyke has basic support for functions and classes. Spyke can do very  
basic type inference for local variables in function bodies. Spyke  
also has
partial support for homogenous lists and dictionaries and fixed length tuples.
One big advantage of Spyke is that it understands at least part of  
numpy. Numpy arrays  are treated as fundamental types and Spyke knows  
what C code to
generate for slicing/indexing of numpy arrays etc. This should help a
lot in scientific applications. Note that Spyke can handle only a
subset of Python. Exceptions, iterators, generators, runtime code
generation of any kind etc is not handled. Nested functions will be  
added soon. I will definitely add some of these missing features based  
on what is actually required for real world Python codes. Currently if  
Spyke does not understand a function, it  just leaves it as Python  
code. Classes can be handled but special
methods are not currently supported. The support of classes is a
little brittle because I am trying to resolve some issues b/w old and
new style of classes.

Where is Spyke?
Spyke will be available as a binary only release in a couple of weeks.
I intend to make it open source after a few months.
Spyke is written in Python and Java and should be platform independant.
I do intend to make the source open in a few months. Right now its
undergoing very rapid development and has negligible amounts of  
documentation so the source code right now is pretty useless to anyone  
else anyway.

I need help:
However I need a bit of help. I am having a couple of problems :
a) I am finding it hard to get pure Python+NumPy testing codes. I need
more codes to test the compiler. Developing a compiler without a  
test-suite is kind of useless. If you have some pure Python codes  
which need better performance, please contact me. I guarantee that  
your codes will not be released to public without your permission but   
might be referenced in academic publications. I can also make the   
compiler available to you hopefully after 10th of April. Its kind of   
unstable currently. I will also need your help in annotating the  
provided testing codes since I probably wont know what your  
application is doing.

b) Libraries which interface with C/C++ : Many codes in SciPy for
instance have mixed language codes. Part of the code is written in
C/C++. Spyke only knows how to annotated Python codes. For C/C++
libraries wrapped into Python modules, Spyke will therefore need to
know at least 2 things :
i) The mapping of a C function name/struct etc  to Python
ii) The type information of the said C function.

There are many many ways that people interact with C code. People
either write wrappers manually, or use autogenerated wrappers using
SWIG or SIP Boost.Python etc., use Pyrex or Cython while some people
use ctypes. I dont have the time or resources to support these
multitude of methods. I considered trying to parse the C code  
implementing wrappers but its "non-trivial" to put it mildly. Parsing  
only SWIG generated code is another possibility but its still hard.  
Another approach that I am seriously considering is to support a  
subset of ctypes (with additional restriction) instead. But my  
question is : Is  ctypes good enough for most of you? Ctypes cannot  
interface with C++  code but its pure Python. However I have not seen  
too many instances of people using ctypes.

c) Strings as type declarations : Do you think I should use decorators
instead at least for function type declarations?

thanks for patiently reading this,
comments and inquiries sought.
rahul

From theller at ctypes.org  Mon Apr  7 08:45:43 2008
From: theller at ctypes.org (Thomas Heller)
Date: Mon, 07 Apr 2008 08:45:43 +0200
Subject: [Python-Dev] Remove "current" Windows executables from
 Lib/distutils/command in svn?
In-Reply-To: <47F86A84.6000204@v.loewis.de>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
Message-ID: <47F9C317.9060701@ctypes.org>

Martin v. L?wis schrieb:
>> I'd like to propose we delete Lib/Distutils/command/wininst-9.0.exe, and
>> enable the building of that project by default in the standard build process
>> (and I'll setup the x64 build of the executable similarly).
> 
> There are two issues here:
> a) how does the binary get into the release tarball? You might argue
>    that it doesn't have to, as it is sufficient when it is included in
>    the Windows installer, however, as currently implemented,
>    bdist_wininst also runs on Unix, and people use it that way.
> b) IIRC, upx was used to compress these executables. I don't think the
>    the current build process covers this, yet (but then, the current
>    binaries might not be compressed with upx anymore, either - not
>    sure whether that would be a bug). Of course, upx would probably
>    not currently apply to Win64-amd64.

I did not compress them with upx anymore because of licensing issues
with the upx-created executables, but I do not remember the details anymore.

Thomas


From lists at cheimes.de  Mon Apr  7 11:57:27 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 07 Apr 2008 11:57:27 +0200
Subject: [Python-Dev] Remove "current" Windows executables from
 Lib/distutils/command in svn?
In-Reply-To: <47F86A84.6000204@v.loewis.de>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
Message-ID: <47F9F007.2060903@cheimes.de>

Martin v. L?wis schrieb:
> b) IIRC, upx was used to compress these executables. I don't think the
>    the current build process covers this, yet (but then, the current
>    binaries might not be compressed with upx anymore, either - not
>    sure whether that would be a bug). Of course, upx would probably
>    not currently apply to Win64-amd64.

The 9.0 binary hasn't been compressed with upx (yet).

Christian

From lists at cheimes.de  Mon Apr  7 11:57:27 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 07 Apr 2008 11:57:27 +0200
Subject: [Python-Dev] Remove "current" Windows executables from
 Lib/distutils/command in svn?
In-Reply-To: <47F86A84.6000204@v.loewis.de>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
Message-ID: <47F9F007.2060903@cheimes.de>

Martin v. L?wis schrieb:
> b) IIRC, upx was used to compress these executables. I don't think the
>    the current build process covers this, yet (but then, the current
>    binaries might not be compressed with upx anymore, either - not
>    sure whether that would be a bug). Of course, upx would probably
>    not currently apply to Win64-amd64.

The 9.0 binary hasn't been compressed with upx (yet).

Christian


From draghuram at gmail.com  Mon Apr  7 15:52:42 2008
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Mon, 7 Apr 2008 09:52:42 -0400
Subject: [Python-Dev] socket.SOL_REUSEADDR: different semantics between
	Windows vs Unix (or why test_asynchat is sometimes dying on Windows)
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E226D8C7C@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB291@EXMBX04.exchhosting.com>
	<20080404220153.6859.1817508967.divmod.quotient.25765@ohm>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E226D8C7C@EXMBX04.exchhosting.com>
Message-ID: <2c51ecee0804070652y2bd30d7apcebb4a96fd7685cb@mail.gmail.com>

On Sat, Apr 5, 2008 at 1:22 PM, Trent Nelson <tnelson at onresolve.com> wrote:

>  Nod, if SO_EXCLUSIVEADDRUSE is used instead in the code I posted, Windows raises EADDRINUSE on the second bind().  I don't have access to any Linux boxes at the moment, so I can't test what sort of error is raised with the example I posted if listen() and accept() are called on the two sockets bound to identical addresses.  Can anyone else shed some light on this?  I'd be interested in knowing if the process wedges on Linux as badly as it does on Windows (to the point where it's not respecting ctrl-c or sigkill).

When I call sock1.listen(5) after sock1.bind(), the test passes for me
on SuSE Linux 10.1

Thanks,
Raghu

From tjreedy at udel.edu  Mon Apr  7 18:35:00 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 7 Apr 2008 12:35:00 -0400
Subject: [Python-Dev] New project : Spyke python-to-C compiler
References: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
Message-ID: <ftdifi$s1p$1@ger.gmane.org>


"Rahul Garg" <garg1 at ualberta.ca> wrote in message 
news:20080406184850.xiqr9t0hwgco004s at webmail.ualberta.ca...
| Note this message has been posted to numpy-discussion and python-dev.
| Sorry for the multiple posting but I thought both python devs and
| numpy users will be interested. If you believe your list should not
| receive this email, let me know. Also I just wanted to introduce
| myself since I may ask doubts about Python and Numpy internals from
| time to time :)

Pydev is for discussion about development of  future versions of Python 
(ie, 2.6 and some of 3.0).  I think this would have been better posted on 
comp.lang.python (or gmane.comp.python.general or the corresponding mailing 
list).  You can get answers about current Python internals there. 




From robert.kern at gmail.com  Mon Apr  7 23:14:16 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 07 Apr 2008 14:14:16 -0700
Subject: [Python-Dev] configure error: "rm: conftest.dSYM: is a
	directory"
In-Reply-To: <18424.52783.238721.876293@montanaro-dyndns-org.local>
References: <18423.53311.150471.667891@montanaro-dyndns-org.local>	<bbaeab100804060113w12ddcef7v81a23a97318b3d78@mail.gmail.com>
	<18424.52783.238721.876293@montanaro-dyndns-org.local>
Message-ID: <fte2r8$qmv$1@ger.gmane.org>

skip at pobox.com wrote:
>     >> Note the "rm: conftest.dSYM: is a directory".  This occurred a few
>     >> times during the configure process.  Didn't cause it to conk out, but
>     >> is annoying.
> 
>     Brett> I am assuming this is on your OS X machine, Skip?
> 
> Yes, sorry.  I forgot to mention that.  As long as someone else sees it and
> feels fine to ignore it, I'm fine with it.  I suspect it's a configure
> problem anyway, not a Python problem.

I've seen it with just about every ./configure script on OS X 10.5. It's not 
specific to Python, and I haven't seen it cause any actual problems.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From guido at python.org  Mon Apr  7 23:44:24 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 7 Apr 2008 14:44:24 -0700
Subject: [Python-Dev] Python 3000: Special type for object attributes &
	map keys
In-Reply-To: <C5EA6644-91C1-43E4-B500-391F552F0609@gmail.com>
References: <C5EA6644-91C1-43E4-B500-391F552F0609@gmail.com>
Message-ID: <ca471dc20804071444o41ce14e6n677f24f373c835ed@mail.gmail.com>

Without an implementation and supporting profile data nobody is going
to believe that you can do name lookup faster than with the built-in
dict type in CPython. Note that names seen by the parser are already
interned, so most of what you seem to be proposing is already
implemented...

On Wed, Mar 5, 2008 at 2:27 PM, Henrik Vendelbo <hvendelbo.dev at gmail.com> wrote:
> It appears to me that if you can make mapping mechanisms faster in
>  Python you can make significant
>  overall speed improvements. I also think the proposed concept could
>  add flexibility to persistence formats
>  and RMI interfaces.
>
>  My basic idea is to have a constant string type with an interpreter
>  globally unique hash. If the original constant
>  is created in a manner different from string constants, it can be
>  tracked and handled differently by the interpreter.
>
>  Obviously most object attribute references are done with the dot
>  operator, so I guess the interpreter already has
>  an efficient mapping mechanism. But there must be a crossover with
>  __getattr__ etc, where a map of some sort is
>  used. I imagine that having a global namespace to translate attribute
>  names into integers could be used for several
>  purposes by the interpreter as well as an application exchanging
>  objects with other applications.
>
>  I imagine these expressions to be supported:
>  * attrname(string) - creates an attrname value from the string
>  * int(attrname) - gets the hash value
>  * string(attrname) - gets the string value
>
>  Hope this makes sense
>
>  Henrik
>
>  P.S. I originally thought of this in a JavaScript context so forgive
>  me if this would make little difference in Python.
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From d at 0x1.org  Mon Apr  7 03:51:45 2008
From: d at 0x1.org (David Arnold)
Date: Sun, 6 Apr 2008 21:51:45 -0400
Subject: [Python-Dev] [Distutils] Remove "current" Windows executables
 from	Lib/distutils/command in svn?
In-Reply-To: <47F86A84.6000204@v.loewis.de>
References: <018701c8978e$ce620660$6b261320$@com.au>
	<47F86A84.6000204@v.loewis.de>
Message-ID: <B9E76FE5-5035-48E7-85C3-0496C5D17481@0x1.org>

On 06/04/2008, at 2:15 AM, Martin v. L?wis wrote:

> a) how does the binary get into the release tarball? You might argue
>    that it doesn't have to, as it is sufficient when it is included in
>    the Windows installer, however, as currently implemented,
>    bdist_wininst also runs on Unix, and people use it that way.

It would be a shame to lose this ability.  It's very handy.




d


From avikohn at gmail.com  Mon Apr  7 06:58:40 2008
From: avikohn at gmail.com (Avi Kohn)
Date: Mon, 7 Apr 2008 00:58:40 -0400
Subject: [Python-Dev] python source code
Message-ID: <ac1baaf0804062158y743a49ebide109b7d6c4ace97@mail.gmail.com>

I am interested in understanding the python source code. Can someone direct
me to resources (documentation,book,archive of mailling lists,etc) that will
assist me ?
Thank you,
Avi

-- 
Avi Kohn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080407/8bf71990/attachment.htm 

From dagss at student.matnat.uio.no  Mon Apr  7 11:33:05 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 7 Apr 2008 11:33:05 +0200 (CEST)
Subject: [Python-Dev] [Numpy-discussion] New project : Spyke python-to-C
	compiler
In-Reply-To: <50807.193.157.243.12.1207560672.squirrel@webmail.uio.no>
References: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
	<50807.193.157.243.12.1207560672.squirrel@webmail.uio.no>
Message-ID: <50810.193.157.243.12.1207560785.squirrel@webmail.uio.no>


> (Though as the saying goes, little duplication is normal (and perhaps
> wanted) for open source software.)

Sorry! I meant "a little", completely reversing the meaning of my sentence.

Dag Sverre


From dagss at student.matnat.uio.no  Mon Apr  7 11:31:12 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 7 Apr 2008 11:31:12 +0200 (CEST)
Subject: [Python-Dev] [Numpy-discussion] New project : Spyke python-to-C
	compiler
In-Reply-To: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
References: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
Message-ID: <50807.193.157.243.12.1207560672.squirrel@webmail.uio.no>


> What is Spyke?
> In many performance critical projects, it is often necessary to
> rewrite parts of the application in C. However writing C wrappers can
> be time consuming. Spyke offers an alternative approach. You add
> annotations to your  Python code as strings. These strings are
> discarded by the Python
> interpreter but these are interpreted as types by Spyke compiler to
> convert to C.

Have you had a look at Cython? http://cython.org.

>From seeing what you write, it looks like we have almost exactly the same
long-term goals; one could almost say that the two pieces of software will
be complete duplicates in functionality. (Cython isn't "there" just yet
though.)

(Though as the saying goes, little duplication is normal (and perhaps
wanted) for open source software.)

Dag Sverre


From musiccomposition at gmail.com  Tue Apr  8 01:01:44 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 7 Apr 2008 18:01:44 -0500
Subject: [Python-Dev] python source code
In-Reply-To: <ac1baaf0804062158y743a49ebide109b7d6c4ace97@mail.gmail.com>
References: <ac1baaf0804062158y743a49ebide109b7d6c4ace97@mail.gmail.com>
Message-ID: <1afaf6160804071601q71d43dd0x39c278188373203a@mail.gmail.com>

On Sun, Apr 6, 2008 at 11:58 PM, Avi Kohn <avikohn at gmail.com> wrote:

> I am interested in understanding the python source code. Can someone
> direct me to resources (documentation,book,archive of mailling lists,etc)
> that will assist me ?
>
What part(s) do you want to learn about? The CPython interpreter is quite
huge. I recommend you ask the comp.lang.python mailing list for further
advice.

>  Thank you,
> Avi
>
> --
> Avi Kohn
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>
>


-- 
Cheers,
Benjamin Peterson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080407/07f9d09f/attachment-0001.htm 

From musiccomposition at gmail.com  Tue Apr  8 01:04:52 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 7 Apr 2008 18:04:52 -0500
Subject: [Python-Dev] New project : Spyke python-to-C compiler
In-Reply-To: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
References: <20080406184850.xiqr9t0hwgco004s@webmail.ualberta.ca>
Message-ID: <1afaf6160804071604g1b7f8eeanda6673444b6e88e0@mail.gmail.com>

[snip]
>
>
>
> c) Strings as type declarations : Do you think I should use decorators
> instead at least for function type declarations?

You might be interested in 3.0's (and maybe 2.6's) function annotations. See
PEP 3107.

>
>
> thanks for patiently reading this,
> comments and inquiries sought.
> rahul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080407/8875ab2f/attachment.htm 

From bonelake at gmail.com  Tue Apr  8 01:14:53 2008
From: bonelake at gmail.com (Brad Miller)
Date: Mon, 7 Apr 2008 18:14:53 -0500
Subject: [Python-Dev] string representation of range in 3.0
Message-ID: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>

Hi,

I use Python in my CS1 and CS2 curriculum and I have a question.
As I've been using the Python 3.0 alphas one of the things that I am  
bothered by is that I cannot see the sequence produced by range  
without introducing students to the list() function.

I typically introduce range on day 1 of class and show students what  
it produces without making a big deal out of the fact that it creates  
a list.  They all accept this and things work out nicely when I  
introduce lists for real in a week or two.

My question is why couldn't the __str__ method for the range object be  
more friendly and show a representation of the sequence?  I understand  
why __repr__ should return range(0,10) for an object created using  
range(10) but couldn't print(range(10)) produce [0, 1, 2, ... 9]   
The ... could even be used if the sequence were excessively wrong.

If this is acceptable, I would be happy to accept the challenge of  
providing a patch.

Thanks,

Brad



From guido at python.org  Tue Apr  8 01:24:19 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 7 Apr 2008 16:24:19 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
Message-ID: <ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>

I'd object to it returning something that resembles a list too
closely, but I could live with str(range(3)) return <0, 1, 2>. We
should probably have a cutoff so that if there are more than 6 values
it'll show the first 3 values, then dots, then the last 2 values. (The
cutoff would be computed so that '...' always represents at least 2
values.

On Mon, Apr 7, 2008 at 4:14 PM, Brad Miller <bonelake at gmail.com> wrote:
> Hi,
>
>  I use Python in my CS1 and CS2 curriculum and I have a question.
>  As I've been using the Python 3.0 alphas one of the things that I am
>  bothered by is that I cannot see the sequence produced by range
>  without introducing students to the list() function.
>
>  I typically introduce range on day 1 of class and show students what
>  it produces without making a big deal out of the fact that it creates
>  a list.  They all accept this and things work out nicely when I
>  introduce lists for real in a week or two.
>
>  My question is why couldn't the __str__ method for the range object be
>  more friendly and show a representation of the sequence?  I understand
>  why __repr__ should return range(0,10) for an object created using
>  range(10) but couldn't print(range(10)) produce [0, 1, 2, ... 9]
>  The ... could even be used if the sequence were excessively wrong.
>
>  If this is acceptable, I would be happy to accept the challenge of
>  providing a patch.
>
>  Thanks,
>
>  Brad
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Apr  8 02:37:54 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 7 Apr 2008 17:37:54 -0700
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
Message-ID: <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>

On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On Wed, Mar 19, 2008 at 2:15 PM,  <glyph at divmod.com> wrote:
>  >
>  >  On 02:21 pm, murman at gmail.com wrote:
>  >  >>OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine
>  >  >>with me.
>  >  >
>  >  >This strikes me as a gratuitous API change of the kind Guido was
>  >  >warning about in his recent post: "Don't change your APIs incompatibly
>  >  >when porting to Py3k"
>  >
>  >  I agree emphatically.  Actually I think this is the most extreme case.
>  >  The unit test stuff should be as stable as humanly possible between 2
>  >  and 3, moreso than any other library.
>
>  This is convincing for me. Move my +1 back to 3.1.

Same here; let's tread carefully here and not change this with 3.0.
Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
I like using only the assertKeyword variants, removing assert_, fail*,
and assertEquals. However I don't like changing assertTrue and
assertFalse to insist that the value is exactly True or False -- if
you really care that much, let's add assertIs(x, y) which asserts that
x and y are the same object. I also think that all tests should use
the operator their name implies, e.g. assertEqual(x, y) should do
something like

  if x == y:
    pass
  else:
    raise AssertionError(...)

rather than

  if x != y:
    raise AssertionError(...)

Someone please open a bug for this task.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Tue Apr  8 03:02:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 7 Apr 2008 20:02:08 -0500
Subject: [Python-Dev] unittest's redundant assertions: asserts vs.
	failIf/Unlesses
In-Reply-To: <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
Message-ID: <1afaf6160804071802j28a247f0t70af39044c691e14@mail.gmail.com>

On Mon, Apr 7, 2008 at 7:37 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
>  > On Wed, Mar 19, 2008 at 2:15 PM,  <glyph at divmod.com> wrote:
>  >  >
>  >  >  On 02:21 pm, murman at gmail.com wrote:
>  >  >  >>OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine
>  >  >  >>with me.
>  >  >  >
>  >  >  >This strikes me as a gratuitous API change of the kind Guido was
>  >  >  >warning about in his recent post: "Don't change your APIs incompatibly
>  >  >  >when porting to Py3k"
>  >  >
>  >  >  I agree emphatically.  Actually I think this is the most extreme case.
>  >  >  The unit test stuff should be as stable as humanly possible between 2
>  >  >  and 3, moreso than any other library.
>  >
>  >  This is convincing for me. Move my +1 back to 3.1.
>
>  Same here; let's tread carefully here and not change this with 3.0.
>  Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
>  I like using only the assertKeyword variants, removing assert_, fail*,
>  and assertEquals. However I don't like changing assertTrue and
>  assertFalse to insist that the value is exactly True or False -- if
>  you really care that much, let's add assertIs(x, y) which asserts that
>  x and y are the same object. I also think that all tests should use
>  the operator their name implies, e.g. assertEqual(x, y) should do
>  something like
>
>   if x == y:
>     pass
>   else:
>     raise AssertionError(...)
>
>  rather than
>
>   if x != y:
>     raise AssertionError(...)
>
>  Someone please open a bug for this task.
See 2578.
>
>  --
>  --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson

From fuzzyman at voidspace.org.uk  Tue Apr  8 10:58:46 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 08 Apr 2008 09:58:46 +0100
Subject: [Python-Dev] unittest's redundant assertions: asserts
	vs.	failIf/Unlesses
In-Reply-To: <ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
Message-ID: <47FB33C6.4080008@voidspace.org.uk>

Guido van Rossum wrote:
> On Wed, Mar 19, 2008 at 5:16 PM, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
>   
>> On Wed, Mar 19, 2008 at 2:15 PM,  <glyph at divmod.com> wrote:
>>  >
>>  >  On 02:21 pm, murman at gmail.com wrote:
>>  >  >>OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine
>>  >  >>with me.
>>  >  >
>>  >  >This strikes me as a gratuitous API change of the kind Guido was
>>  >  >warning about in his recent post: "Don't change your APIs incompatibly
>>  >  >when porting to Py3k"
>>  >
>>  >  I agree emphatically.  Actually I think this is the most extreme case.
>>  >  The unit test stuff should be as stable as humanly possible between 2
>>  >  and 3, moreso than any other library.
>>
>>  This is convincing for me. Move my +1 back to 3.1.
>>     
>
> Same here; let's tread carefully here and not change this with 3.0.
> Starting to deprecate in 3.1 and killing in 3.3 would be soon enough.
> I like using only the assertKeyword variants, removing assert_, fail*,
> and assertEquals. However I don't like changing assertTrue and
> assertFalse to insist that the value is exactly True or False -- if
> you really care that much, let's add assertIs(x, y) which asserts that
> x and y are the same object. I also think that all tests should use
> the operator their name implies, e.g. assertEqual(x, y) should do
> something like
>
>   if x == y:
>     pass
>   else:
>     raise AssertionError(...)
>
> rather than
>
>   if x != y:
>     raise AssertionError(...)
>
> Someone please open a bug for this task.
>
>   
This sounds like a good compromise and I'm happy to take on the cleanup 
- unless someone else beats me to it. I guess it should wait until 3.0 
final is out of the door before adding the DeprecationWarnings.

Michael Foord

From tnelson at onresolve.com  Tue Apr  8 14:00:12 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 8 Apr 2008 05:00:12 -0700
Subject: [Python-Dev] Changing all network-oriented tests to facilitate
 (virtually unlimited) parallel test execution [Issue#: 2550]
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EEF@EXMBX04.exchhosting.com>


I've forwarding my most recent update to issue 2550 here such that the proposed patch (and in general, the approach to network-oriented test cases) can be vetted by a wider audience:

http://bugs.python.org/file9980/trunk.2550-2.patch

This patch works towards fixing a large proportion of the tests that were written in such a way that often leads to buildbot errors when port conflicts arise.  With this patch applied, I can run many instances of the test suite in parallel and none of the network-oriented tests fail (which they do currently if you try and do this).

Plenty of discussion (mostly me replying to my own comments) on the subject at: http://bugs.python.org/issue2550.

Anyone have any issues with this new approach?  I'm particularily interested in whether or not people disagree with the assumption I've drawn regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below).

    Trent.


________________________________________
From: python-bugs-list-bounces+tnelson=onresolve.com at python.org [python-bugs-list-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [report at bugs.python.org]
Sent: 08 April 2008 12:49
To: python-bugs-list at python.org
Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on Windows    as on Unix

Trent Nelson <tnelson at onresolve.com> added the comment:

Invested quite a few cycles on this issue last night.  The more time I
spent on it, the more I became convinced that every single test working
with sockets should be changed in one fell swoop in order to facilitate
(virtually unlimited) parallel test execution without fear of port
conflicts.

I've attached a second patch, trunk.2550-2.patch, which is my progress
so far on doing just this.  The main changes can be expressed by the
following two points:

a) do whatever it takes in network-oriented tests to ensure
   unique ports are obtained (relying on the bind_port() and
   find_unused_port() methods exposed by test_support)

b) never, ever, ever call SO_REUSEADDR on a socket from a test;
   because we're putting so much effort into obtaining a unique
   port, this should never be necessary -- in the rare cases that
   our attempts to obtain a unique port fail, then we absolutely
   should fail with EADDRINUSE, as the ability to obtain a unique
   port for the duration of a client/server test is an invariant
   that we *must* be able to depend upon.  If the invariant is
   broken, fail immediately (don't mask the problem with
   SO_REUSEADDR).

With this patch applied, I can spawn a handful of Python processes and
run the entire test suite (without -r, ensuring all tests are run in
the same order, which should encourage port conflicts (if there were
any)) without any errors*.  Doing that now is completely and utterly
impossible.

[*] Well, almost without error.  All the I/O related tests that try and
open @test fail.

I believe there's still outstanding work to do with this patch with
regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE
should be handled in the rest of the stdlib.  I'm still thinking about
the best approach for this.  However, the patch as it currently stands
is still quite substantial so I wanted to get it out sooner rather than
later for review.

(I'll forward this to python-dev@ to try and encourage more eyes from
people with far more network-fu than I.)

Added file: http://bugs.python.org/file9980/trunk.2550-2.patch

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2550>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve.com

From alessandro.guido at gmail.com  Tue Apr  8 15:22:50 2008
From: alessandro.guido at gmail.com (Alessandro Guido)
Date: Tue, 8 Apr 2008 15:22:50 +0200
Subject: [Python-Dev] py3k: print function treats sep=None and end=None in
	an unintuitive way
Message-ID: <200804081522.50764.alessandro.guido@gmail.com>

Can anybody please point me why print('a', 'b', sep=None, end=None) should
produce "a b\n" instead of "ab"?
I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
ml threads but did not find a good reason justifying such a strange behaviour.

Thanks.

	-Alessandro Guido

From ncoghlan at gmail.com  Tue Apr  8 15:57:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 08 Apr 2008 23:57:08 +1000
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in	an unintuitive way
In-Reply-To: <200804081522.50764.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
Message-ID: <47FB79B4.5020605@gmail.com>

Alessandro Guido wrote:
> Can anybody please point me why print('a', 'b', sep=None, end=None) should
> produce "a b\n" instead of "ab"?
> I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
> ml threads but did not find a good reason justifying such a strange behaviour.

So that print(a, b) does the right thing (i.e. matches the Python 2.x 
print statement's behaviour)

sep=None, end=None means "use the default separator and line ending" 
rather than "don't use a separator or line ending" (this is the same as 
for most functions with optional arguments - you need to look at the 
documentation of the function to find out what default values are used 
when passing None for an optional argument).

That said, it does read oddly, so I'd advise against writing it out 
explicitly like that - if you want the default, just leave out the 
relevant argument.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From eric+python-dev at trueblade.com  Tue Apr  8 15:38:26 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 08 Apr 2008 09:38:26 -0400
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in	an unintuitive way
In-Reply-To: <200804081522.50764.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
Message-ID: <47FB7552.4020109@trueblade.com>

Alessandro Guido wrote:
> Can anybody please point me why print('a', 'b', sep=None, end=None) should
> produce "a b\n" instead of "ab"?
> I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
> ml threads but did not find a good reason justifying such a strange behaviour.
> 
> Thanks.
> 
> 	-Alessandro Guido

Because None mean 'use the default value'.  You probably want:
print('a', 'b', sep='', end='')

From eric+python-dev at trueblade.com  Tue Apr  8 15:40:26 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 08 Apr 2008 09:40:26 -0400
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in	an unintuitive way
In-Reply-To: <200804081522.50764.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
Message-ID: <47FB75CA.6070601@trueblade.com>

Alessandro Guido wrote:
> Can anybody please point me why print('a', 'b', sep=None, end=None) should
> produce "a b\n" instead of "ab"?
> I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
> ml threads but did not find a good reason justifying such a strange behaviour.
> 
> Thanks.
> 
> 	-Alessandro Guido

Because None mean 'use the default value'.  You probably want:
print('a', 'b', sep='', end='')

From eric+python-dev at trueblade.com  Tue Apr  8 15:44:02 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 08 Apr 2008 09:44:02 -0400
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in	an unintuitive way
In-Reply-To: <200804081522.50764.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
Message-ID: <47FB76A2.7020600@trueblade.com>

Alessandro Guido wrote:
> Can anybody please point me why print('a', 'b', sep=None, end=None) should
> produce "a b\n" instead of "ab"?
> I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
> ml threads but did not find a good reason justifying such a strange behaviour.
> 
> Thanks.
> 
> 	-Alessandro Guido

Because None means 'use the default value'.  You probably want:
print('a', 'b', sep='', end='')

 >>> import io
 >>> s = io.StringIO()
 >>> print('a', 'b', end='', sep='', file=s)
 >>> s.getvalue()
'ab'
 >>>

From eric+python-dev at trueblade.com  Tue Apr  8 16:22:45 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 08 Apr 2008 10:22:45 -0400
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in	an unintuitive way
In-Reply-To: <47FB76A2.7020600@trueblade.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
	<47FB76A2.7020600@trueblade.com>
Message-ID: <47FB7FB5.4010506@trueblade.com>

[Sorry for the dupes.  Lesson: never try and send mail from a moving train.]

Eric Smith wrote:
> Alessandro Guido wrote:
>> Can anybody please point me why print('a', 'b', sep=None, end=None) should
>> produce "a b\n" instead of "ab"?
>> I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
>> ml threads but did not find a good reason justifying such a strange behaviour.
>>
>> Thanks.
>>
>> 	-Alessandro Guido
> 
> Because None means 'use the default value'.  You probably want:
> print('a', 'b', sep='', end='')
> 
>  >>> import io
>  >>> s = io.StringIO()
>  >>> print('a', 'b', end='', sep='', file=s)
>  >>> s.getvalue()
> 'ab'
>  >>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/eric%2Bpython-dev%40trueblade.com
> 


From tnelson at onresolve.com  Tue Apr  8 16:47:43 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 8 Apr 2008 07:47:43 -0700
Subject: [Python-Dev] [OT] Wingware IDE key for sprinters at PyCon
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EF5@EXMBX04.exchhosting.com>

Anyone happen to have the key handy that Wingware were giving out to sprinters at PyCon?  For the life of me, I can't find what I did with that piece of paper.  If someone could forward me it off list, that'd be great.

    Trent.

From alessandro.guido at gmail.com  Tue Apr  8 18:01:15 2008
From: alessandro.guido at gmail.com (Alessandro Guido)
Date: Tue, 8 Apr 2008 18:01:15 +0200
Subject: [Python-Dev]
	=?iso-8859-1?q?py3k=3A_print_function_treats_sep=3DN?=
	=?iso-8859-1?q?one_and_end=3DNone_in=09an_unintuitive_way?=
In-Reply-To: <47FB79B4.5020605@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
	<47FB79B4.5020605@gmail.com>
Message-ID: <200804081801.15687.alessandro.guido@gmail.com>

Nick Coghlan wrote:
> So that print(a, b) does the right thing (i.e. matches the Python 2.x 
> print statement's behaviour)

AFAICS print(a, b) does the right thing because default values of "sep" and
"end" are ' ' and '\n' respectively, doesn't it?

Eric Smith wrote:
> Because None means 'use the default value'.  You probably want:
> print('a', 'b', sep='', end='')

I think this is a "not optimally designed" API
because you have to read the documentation to understand why

print('a', 'b', sep=None, end=None)

does not translate into the obvious:

?print strings 'a' and 'b' using no separator and no terminator?

but into:

?print strings 'a' and 'b' using the default separator and the default terminator?

However i'll just cope with it, Python is still the best language ;)

Thank you all for replying!

	-Alessandro Guido

From zooko at zooko.com  Tue Apr  8 19:01:23 2008
From: zooko at zooko.com (zooko)
Date: Tue, 8 Apr 2008 10:01:23 -0700
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are	 good for you
In-Reply-To: <47EB07A6.6040800@plope.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
Message-ID: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>


On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> zooko wrote:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

>> Here is a simple proposal:  make the standard Python "import"   
>> mechanism notice eggs on the PYTHONPATH and insert them (into the   
>> *same* location) on the sys.path.
>> This eliminates the #1 problem with eggs -- that they don't  
>> easily  work when installing them into places other than your site- 
>> packages  and that if you allow any of them to be installed on  
>> your system then  they take precedence over your non-egg packages  
>> even you explicitly  put those other packages earlier in your  
>> PYTHONPATH.  (That latter  behavior is very disagreeable to more  
>> than a few prorgammers.)
>
> Sorry if I'm out of the loop and there's some subtlety here that  
> I'm disregarding, but it doesn't appear that either of the issues  
> you mention is a actually problem with eggs.  These are instead  
> problems with how eggs get installed by easy_install (which uses  
> a .pth file to extend sys.path).  It's reasonable to put eggs on  
> the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))  
> instead of using easy_install to install them.

Yes, you are missing something.  While many programmers, such as  
yourself and Lennart Regebro (who posted to this thread) find the  
current eggs system to be perfectly convenient and to Just Work, many  
others, such as Glyph Lefkowitz (who posted to a related thread) find  
them to be so annoying that they actively ensure that no eggs are  
ever allowed to touch their system.

The reasons for this latter problem are two:

1.  You can't conveniently install eggs into a non-system directory,  
such as ~/my-python-stuff.

2.  If you allow even a single egg to be installed into your  
PYTHONPATH, it will change the semantics of your PYTHONPATH.

Both of these problems are directly caused by the need for eggs to  
hack your site.py.  If Python automatically added eggs found in the  
PYTHONPATH to the sys.path, both of these problems would go away.

I am skeptical that the current proposals to define a new database  
for installed packages will fare any better than the current eggs  
scheme does in this respect.

This issue is important to me, because the benefits of eggs grow  
superlinearly with the number of good programmers who use them.  They  
are a tool for re-using source code -- a tool for cooperation between  
programmers.  To gain the greatest benefits at this point we do not  
need to add new features to eggs, we need to make them more palatable  
to more good programmers.

I am skeptical that prorgammers are going to be willing to use a new  
database format.  They already have a database -- their filesystem --  
and they already have the tools to control it -- mv, rm, and  
PYTHONPATH.  Many of them already hate the existence the  
"easy_instlal.pth" database file, and I don't see why a new database  
file would be any different.

My proposal makes the current benefits of eggs -- clean, easy code re- 
use among programmers -- more compatible with their current tools --  
mv, rm, and PYTHONPATH.  It is also forward-compatible with more  
sophisticated proposals to add features like uninstall and operating  
system integration.

By the way, since I posted my proposal two weeks ago I have pointed a  
couple of Python hackers who currently refuse to use eggs at the URL:

http://mail.python.org/pipermail/python-dev/2008-March/078243.html

They both agreed that it made perfect sense.  I told one of them  
about the alternate proposal to define a new database file to contain  
a list of installed packages, and he sighed and rolled his eyes and  
said "So they are planning to reinvent apt!".

Regards,

Zooko


From ncoghlan at gmail.com  Tue Apr  8 19:15:58 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 09 Apr 2008 03:15:58 +1000
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in an unintuitive way
In-Reply-To: <200804081801.15687.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>
	<47FB79B4.5020605@gmail.com>
	<200804081801.15687.alessandro.guido@gmail.com>
Message-ID: <47FBA84E.3050608@gmail.com>

Alessandro Guido wrote:
> ?print strings 'a' and 'b' using the default separator and the default terminator?
> 
> However i'll just cope with it, Python is still the best language ;)

I definitely recommend getting used to this idiom - None is used to 
indicate missing (i.e. 'use the default value') arguments in many more 
cases than just the print function.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Tue Apr  8 19:42:08 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 08 Apr 2008 13:42:08 -0400
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are	 good for you
In-Reply-To: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <20080408174307.79E453A4108@sparrow.telecommunity.com>

At 10:01 AM 4/8/2008 -0700, zooko wrote:

>On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
> > zooko wrote:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
> >> Here is a simple proposal:  make the standard Python "import"
> >> mechanism notice eggs on the PYTHONPATH and insert them (into the
> >> *same* location) on the sys.path.
> >> This eliminates the #1 problem with eggs -- that they don't
> >> easily  work when installing them into places other than your site-
> >> packages  and that if you allow any of them to be installed on
> >> your system then  they take precedence over your non-egg packages
> >> even you explicitly  put those other packages earlier in your
> >> PYTHONPATH.  (That latter  behavior is very disagreeable to more
> >> than a few prorgammers.)
> >
> > Sorry if I'm out of the loop and there's some subtlety here that
> > I'm disregarding, but it doesn't appear that either of the issues
> > you mention is a actually problem with eggs.  These are instead
> > problems with how eggs get installed by easy_install (which uses
> > a .pth file to extend sys.path).  It's reasonable to put eggs on
> > the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg'))
> > instead of using easy_install to install them.
>
>Yes, you are missing something.  While many programmers, such as
>yourself and Lennart Regebro (who posted to this thread) find the
>current eggs system to be perfectly convenient and to Just Work, many
>others, such as Glyph Lefkowitz (who posted to a related thread) find
>them to be so annoying that they actively ensure that no eggs are
>ever allowed to touch their system.
>
>The reasons for this latter problem are two:
>
>1.  You can't conveniently install eggs into a non-system directory,
>such as ~/my-python-stuff.

Wha?

>2.  If you allow even a single egg to be installed into your
>PYTHONPATH, it will change the semantics of your PYTHONPATH.

Only in the same way that manually putting an egg on the front of 
PYTHONPATH can be considered to "change the semantics" of your PYTHONPATH.


>Both of these problems are directly caused by the need for eggs to
>hack your site.py.  If Python automatically added eggs found in the
>PYTHONPATH to the sys.path, both of these problems would go away.

And add new ones.


>I am skeptical that the current proposals to define a new database
>for installed packages will fare any better than the current eggs
>scheme does in this respect.

The purpose for the installation database is to allow easy_install to 
eschew the use of .egg files or directories for anything other than 
multi-version installs.  Thus, no need to add those .egg files or 
directories to the head of the PYTHONPATH.  Conflicts would be 
handled at install time rather than runtime, in other words.


>I am skeptical that prorgammers are going to be willing to use a new
>database format.  They already have a database -- their filesystem --
>and they already have the tools to control it -- mv, rm, and
>PYTHONPATH.  Many of them already hate the existence the
>"easy_instlal.pth" database file, and I don't see why a new database
>file would be any different.

PEP 262 does not propose a database file -- it proposes the inclusion 
of a metadata file for each installed distribution.


>My proposal makes the current benefits of eggs -- clean, easy code re-
>use among programmers -- more compatible with their current tools --
>mv, rm, and PYTHONPATH.  It is also forward-compatible with more
>sophisticated proposals to add features like uninstall and operating
>system integration.

Actually, your current proposal doesn't work, unless you at least 
have some way to indicate which *version* of an egg should be 
automatically added to sys.path -- and some way to change 
that.  Otherwise, you might as well use the -m option to 
easy_install, and require() the eggs at runtime.  (Which needs 
neither .pth files nor site.py hacking.)

Meanwhile, my understanding is that the people who dislike eggs, 
dislike them because when they install a setuptools-based package, 
it's installed as an egg by default.  The installation database 
proposal (and by the way, people really should read and understand 
PEP 262, including the open issues, before trying to compete with 
it), will allow setuptools-based packages to install the 
"old-fashioned" way by default.  That is, not as eggs.  Similarly, 
easy_install would be able to skip installing .eggs unless you wanted 
multi-version support.

So, people who don't like eggs would never see them, since the only 
way you'd ever get them would be via easy_install -m, and they would 
never use it.


>By the way, since I posted my proposal two weeks ago I have pointed a
>couple of Python hackers who currently refuse to use eggs at the URL:
>
>http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
>They both agreed that it made perfect sense.  I told one of them
>about the alternate proposal to define a new database file to contain
>a list of installed packages, and he sighed and rolled his eyes and
>said "So they are planning to reinvent apt!".

No, we're planning to make it possible for easy_install not to 
overwrite things that would break your system, and allow distutils 
and setuptools to uninstall what they installed.  That's a 
considerably less ambitious goal, by far.  :)


From p.f.moore at gmail.com  Tue Apr  8 19:43:55 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 8 Apr 2008 18:43:55 +0100
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are good for you
In-Reply-To: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <79990c6b0804081043ld93ea2fua01b9eb96be9f5c@mail.gmail.com>

On 08/04/2008, zooko <zooko at zooko.com> wrote:
>  By the way, since I posted my proposal two weeks ago I have pointed a
>  couple of Python hackers who currently refuse to use eggs at the URL:
>
>  http://mail.python.org/pipermail/python-dev/2008-March/078243.html
>
>  They both agreed that it made perfect sense.  I told one of them
>  about the alternate proposal to define a new database file to contain
>  a list of installed packages, and he sighed and rolled his eyes and
>  said "So they are planning to reinvent apt!".

I do think that a simple solution like that has some merit. It has two
significant downsides, however:

1. It requires that core Python "bless" the egg format to some extent
- something Guido has said he is unwilling to do.
2. It ignores the issue of package management completely. Personally,
I avoid anything that doesn't integrate with a unified package manager
(whether that be the Windows add/remove feature, or an
as-yet-to-be-built custom Python package manager is not important).

Filesystem commands do not a package manager make...

Paul.

From chrism at plope.com  Tue Apr  8 19:52:19 2008
From: chrism at plope.com (Chris McDonough)
Date: Tue, 08 Apr 2008 13:52:19 -0400
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are	 good for you
In-Reply-To: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <47FBB0D3.2090405@plope.com>

zooko wrote:
> 
> On Mar 26, 2008, at 7:34 PM, Chris McDonough wrote:
>> zooko wrote:
> 
> http://mail.python.org/pipermail/python-dev/2008-March/078243.html
> 
>>> Here is a simple proposal:  make the standard Python "import"  
>>> mechanism notice eggs on the PYTHONPATH and insert them (into the  
>>> *same* location) on the sys.path.
>>> This eliminates the #1 problem with eggs -- that they don't easily  
>>> work when installing them into places other than your site-packages  
>>> and that if you allow any of them to be installed on your system 
>>> then  they take precedence over your non-egg packages even you 
>>> explicitly  put those other packages earlier in your PYTHONPATH.  
>>> (That latter  behavior is very disagreeable to more than a few 
>>> prorgammers.)
>>
>> Sorry if I'm out of the loop and there's some subtlety here that I'm 
>> disregarding, but it doesn't appear that either of the issues you 
>> mention is a actually problem with eggs.  These are instead problems 
>> with how eggs get installed by easy_install (which uses a .pth file to 
>> extend sys.path).  It's reasonable to put eggs on the PYTHONPATH 
>> manually (e.g. sys.path.append('/path/to/some.egg')) instead of using 
>> easy_install to install them.
> 
> Yes, you are missing something.  While many programmers, such as 
> yourself and Lennart Regebro (who posted to this thread) find the 
> current eggs system to be perfectly convenient and to Just Work, many 
> others, such as Glyph Lefkowitz (who posted to a related thread) find 
> them to be so annoying that they actively ensure that no eggs are ever 
> allowed to touch their system.
> 
> The reasons for this latter problem are two:
> 
> 1.  You can't conveniently install eggs into a non-system directory, 
> such as ~/my-python-stuff.

That's just not true.

$ export PYTHONPATH=/home/you/my-python-stuff/foo-1.3.egg
$ python
 >>> import foo

Eggs are directories (or zipfiles) that contain packages.  They happen to 
contain other metadata directories too, but these can be ignored if you just 
want to *use* them (as opposed to wanting to introspect metadata about them).

> 2.  If you allow even a single egg to be installed into your PYTHONPATH, 
> it will change the semantics of your PYTHONPATH.

I think you are mistaken.  The use of the .pth file that changes sys.path is a 
feature of easy_install, not of eggs.  You don't need to use any .pth file to 
put eggs on your PYTHONPATH.

Note that zc.buildout is a framework that installs eggs, and it doesn't rely at 
all on .pth files to automatically hack sys.path.  Instead, it munges console 
scripts to add each egg dir to sys.path.  This is pretty nasty too, but it does 
prove the point.

It is however true that you need to change sys.path to use an egg.  Is that what 
you're objecting to fundamentally?  You just don't want to have to change 
sys.path at all to use an egg?  Maybe you're objecting to the notion that an egg 
can contain more than one package.  There is functionally no difference between 
an egg and a directory full of packages.

> Both of these problems are directly caused by the need for eggs to hack 
> your site.py.  If Python automatically added eggs found in the 
> PYTHONPATH to the sys.path, both of these problems would go away.

I'm not sure what you mean.  Eggs don't hack site.py.  Eggs are just a packaging 
format.  easy_install doesn't hack site.py either.  It just puts a .pth file 
(the parsing of which is a feature of "core" Python itself, not any setuptools 
magic) in site packages and manages it.

It seems like you're advocating adding magic that you can't turn off (magical 
detection of eggs in an already site.py-approved packages directory) to defeat 
magic that you can turn off (by not using easy_install or .pth files).  At some 
level the magic you want to see built in to Python would almost certainly wind 
up doing what you hate by hacking sys.path unless you wanted to restrict eggs to 
containing a single package only.  And you wouldn't be able to turn it off 
except through some obscure environment variable setting.

> By the way, since I posted my proposal two weeks ago I have pointed a 
> couple of Python hackers who currently refuse to use eggs at the URL:
> 
> http://mail.python.org/pipermail/python-dev/2008-March/078243.html
> 
> They both agreed that it made perfect sense.  I told one of them about 
> the alternate proposal to define a new database file to contain a list 
> of installed packages, and he sighed and rolled his eyes and said "So 
> they are planning to reinvent apt!".

I think changing the Python core is the worst possible answer to this problem. 
"Don't use easy_install" is currently the best, AFAICT.

- C


From tnelson at onresolve.com  Tue Apr  8 20:36:47 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 8 Apr 2008 11:36:47 -0700
Subject: [Python-Dev] [OT] Wingware IDE key for sprinters at PyCon
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EF5@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EF5@EXMBX04.exchhosting.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E226D9B7E@EXMBX04.exchhosting.com>


All sorted, thanks.

> -----Original Message-----
> From: python-dev-bounces+tnelson=onresolve.com at python.org
> [mailto:python-dev-bounces+tnelson=onresolve.com at python.org]
> On Behalf Of Trent Nelson
> Sent: 08 April 2008 15:48
> To: python-dev at python.org
> Subject: [Python-Dev] [OT] Wingware IDE key for sprinters at PyCon
>
> Anyone happen to have the key handy that Wingware were giving
> out to sprinters at PyCon?  For the life of me, I can't find
> what I did with that piece of paper.  If someone could
> forward me it off list, that'd be great.
>
>     Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tnelson%40on
resolve.com
>

From zooko at zooko.com  Tue Apr  8 20:46:27 2008
From: zooko at zooko.com (zooko)
Date: Tue, 8 Apr 2008 11:46:27 -0700
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are	 good for you
In-Reply-To: <1207679260.3474.74.camel@localhost.localdomain>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<1207679260.3474.74.camel@localhost.localdomain>
Message-ID: <236B9285-BE59-45CA-B727-74B3B3B764BA@zooko.com>

On Apr 8, 2008, at 11:27 AM, Lloyd Kvam wrote:
>
> When I wear my sysadmin hat, eggs become a nuisance.
...
> As a developer, eggs are great.
...
> Fortunately, distutils includes tools like bdist_rpm so that python
> modules can be packaged for easy processing by the system package
> manager.  So once I need to switch back to a sysadmin role, I can use
> the system tools to install and track packages.

This is the same experience I have.  I rely on setuptools and eggs  
extensively in developing our software, and I use setuptools and eggs  
as the primary method of giving our source code to other programmers.

But no software is ever installed on our production servers unless  
that software is in .deb form in an apt-gettable repository, and this  
requirement is unlikely to change for the forseeable future.

Regards,

Zooko


From greg at krypto.org  Tue Apr  8 21:58:01 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 8 Apr 2008 12:58:01 -0700
Subject: [Python-Dev] Changing all network-oriented tests to facilitate
	(virtually unlimited) parallel test execution [Issue#: 2550]
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EEF@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EEF@EXMBX04.exchhosting.com>
Message-ID: <52dc1c820804081258r73d5491chf0a79f0f7de8c3f4@mail.gmail.com>

On Tue, Apr 8, 2008 at 5:00 AM, Trent Nelson <tnelson at onresolve.com> wrote:

>
> I've forwarding my most recent update to issue 2550 here such that the
> proposed patch (and in general, the approach to network-oriented test cases)
> can be vetted by a wider audience:
>
> http://bugs.python.org/file9980/trunk.2550-2.patch
>
> This patch works towards fixing a large proportion of the tests that were
> written in such a way that often leads to buildbot errors when port
> conflicts arise.  With this patch applied, I can run many instances of the
> test suite in parallel and none of the network-oriented tests fail (which
> they do currently if you try and do this).
>
> Plenty of discussion (mostly me replying to my own comments) on the
> subject at: http://bugs.python.org/issue2550.
>
> Anyone have any issues with this new approach?  I'm particularily
> interested in whether or not people disagree with the assumption I've drawn
> regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below).
>
>    Trent.


I think your assumptions are fair.

Note that not using SO_REUSEADDR can reserve the port for a few minutes
beyond the actual unbinding destruction of the server socket on some OSes
but with tens of thousands of ports available and likely not more than 1-200
being needed by a full test suite run and how long the test suite takes to
run I think that is acceptable and is not a problem we're likely to ever run
into (fix it only iffwe ever do).

-gps


>
>
>
> ________________________________________
> From: python-bugs-list-bounces+tnelson=onresolve.com at python.org[python-bugs-list-bounces+tnelson=
> onresolve.com at python.org] On Behalf Of Trent Nelson [
> report at bugs.python.org]
> Sent: 08 April 2008 12:49
> To: python-bugs-list at python.org
> Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on
> Windows    as on Unix
>
> Trent Nelson <tnelson at onresolve.com> added the comment:
>
> Invested quite a few cycles on this issue last night.  The more time I
> spent on it, the more I became convinced that every single test working
> with sockets should be changed in one fell swoop in order to facilitate
> (virtually unlimited) parallel test execution without fear of port
> conflicts.
>
> I've attached a second patch, trunk.2550-2.patch, which is my progress
> so far on doing just this.  The main changes can be expressed by the
> following two points:
>
> a) do whatever it takes in network-oriented tests to ensure
>   unique ports are obtained (relying on the bind_port() and
>   find_unused_port() methods exposed by test_support)
>
> b) never, ever, ever call SO_REUSEADDR on a socket from a test;
>   because we're putting so much effort into obtaining a unique
>   port, this should never be necessary -- in the rare cases that
>   our attempts to obtain a unique port fail, then we absolutely
>   should fail with EADDRINUSE, as the ability to obtain a unique
>   port for the duration of a client/server test is an invariant
>   that we *must* be able to depend upon.  If the invariant is
>   broken, fail immediately (don't mask the problem with
>   SO_REUSEADDR).
>
> With this patch applied, I can spawn a handful of Python processes and
> run the entire test suite (without -r, ensuring all tests are run in
> the same order, which should encourage port conflicts (if there were
> any)) without any errors*.  Doing that now is completely and utterly
> impossible.
>
> [*] Well, almost without error.  All the I/O related tests that try and
> open @test fail.
>
> I believe there's still outstanding work to do with this patch with
> regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE
> should be handled in the rest of the stdlib.  I'm still thinking about
> the best approach for this.  However, the patch as it currently stands
> is still quite substantial so I wanted to get it out sooner rather than
> later for review.
>
> (I'll forward this to python-dev@ to try and encourage more eyes from
> people with far more network-fu than I.)
>
> Added file: http://bugs.python.org/file9980/trunk.2550-2.patch
>
> __________________________________
> Tracker <report at bugs.python.org>
> <http://bugs.python.org/issue2550>
> __________________________________
> _______________________________________________
> Python-bugs-list mailing list
> Unsubscribe:
> http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080408/537bb057/attachment-0001.htm 

From greg.ewing at canterbury.ac.nz  Wed Apr  9 01:36:20 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Apr 2008 11:36:20 +1200
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are good for you
In-Reply-To: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <47FC0174.3070605@canterbury.ac.nz>

zooko wrote:

> 1.  You can't conveniently install eggs into a non-system directory,  
> such as ~/my-python-stuff.
> 
> 2.  If you allow even a single egg to be installed into your  
> PYTHONPATH, it will change the semantics of your PYTHONPATH.

I discovered another annoyance with eggs the other day -- it
seems that tracebacks referring to egg-resident files contain the
pathname of some temporary directory that existed when the egg
was being packaged, rather than the one it actually exists in
at run time.

-- 
Greg

From tnelson at onresolve.com  Wed Apr  9 02:01:26 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 8 Apr 2008 17:01:26 -0700
Subject: [Python-Dev] Changing all network-oriented tests to facilitate
 (virtually unlimited) parallel test execution [Issue#: 2550]
In-Reply-To: <52dc1c820804081258r73d5491chf0a79f0f7de8c3f4@mail.gmail.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22939EEF@EXMBX04.exchhosting.com>
	<52dc1c820804081258r73d5491chf0a79f0f7de8c3f4@mail.gmail.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB575@EXMBX04.exchhosting.com>

Committed the patch in r62234.  Hopefully the work paid off!  (He says moments before all the buildbots turn red...)

________________________________
From: Gregory P. Smith [mailto:greg at krypto.org]
Sent: 08 April 2008 20:58
To: Trent Nelson
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Changing all network-oriented tests to facilitate (virtually unlimited) parallel test execution [Issue#: 2550]



On Tue, Apr 8, 2008 at 5:00 AM, Trent Nelson <tnelson at onresolve.com<mailto:tnelson at onresolve.com>> wrote:

I've forwarding my most recent update to issue 2550 here such that the proposed patch (and in general, the approach to network-oriented test cases) can be vetted by a wider audience:

http://bugs.python.org/file9980/trunk.2550-2.patch

This patch works towards fixing a large proportion of the tests that were written in such a way that often leads to buildbot errors when port conflicts arise.  With this patch applied, I can run many instances of the test suite in parallel and none of the network-oriented tests fail (which they do currently if you try and do this).

Plenty of discussion (mostly me replying to my own comments) on the subject at: http://bugs.python.org/issue2550.

Anyone have any issues with this new approach?  I'm particularily interested in whether or not people disagree with the assumption I've drawn regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below).

   Trent.

I think your assumptions are fair.

Note that not using SO_REUSEADDR can reserve the port for a few minutes beyond the actual unbinding destruction of the server socket on some OSes but with tens of thousands of ports available and likely not more than 1-200 being needed by a full test suite run and how long the test suite takes to run I think that is acceptable and is not a problem we're likely to ever run into (fix it only iffwe ever do).

-gps




________________________________________
From: python-bugs-list-bounces+tnelson=onresolve.com at python.org<mailto:onresolve.com at python.org> [python-bugs-list-bounces+tnelson=onresolve.com at python.org<mailto:onresolve.com at python.org>] On Behalf Of Trent Nelson [report at bugs.python.org<mailto:report at bugs.python.org>]
Sent: 08 April 2008 12:49
To: python-bugs-list at python.org<mailto:python-bugs-list at python.org>
Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on Windows    as on Unix

Trent Nelson <tnelson at onresolve.com<mailto:tnelson at onresolve.com>> added the comment:

Invested quite a few cycles on this issue last night.  The more time I
spent on it, the more I became convinced that every single test working
with sockets should be changed in one fell swoop in order to facilitate
(virtually unlimited) parallel test execution without fear of port
conflicts.

I've attached a second patch, trunk.2550-2.patch, which is my progress
so far on doing just this.  The main changes can be expressed by the
following two points:

a) do whatever it takes in network-oriented tests to ensure
  unique ports are obtained (relying on the bind_port() and
  find_unused_port() methods exposed by test_support)

b) never, ever, ever call SO_REUSEADDR on a socket from a test;
  because we're putting so much effort into obtaining a unique
  port, this should never be necessary -- in the rare cases that
  our attempts to obtain a unique port fail, then we absolutely
  should fail with EADDRINUSE, as the ability to obtain a unique
  port for the duration of a client/server test is an invariant
  that we *must* be able to depend upon.  If the invariant is
  broken, fail immediately (don't mask the problem with
  SO_REUSEADDR).

With this patch applied, I can spawn a handful of Python processes and
run the entire test suite (without -r, ensuring all tests are run in
the same order, which should encourage port conflicts (if there were
any)) without any errors*.  Doing that now is completely and utterly
impossible.

[*] Well, almost without error.  All the I/O related tests that try and
open @test fail.

I believe there's still outstanding work to do with this patch with
regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE
should be handled in the rest of the stdlib.  I'm still thinking about
the best approach for this.  However, the patch as it currently stands
is still quite substantial so I wanted to get it out sooner rather than
later for review.

(I'll forward this to python-dev@ to try and encourage more eyes from
people with far more network-fu than I.)

Added file: http://bugs.python.org/file9980/trunk.2550-2.patch

__________________________________
Tracker <report at bugs.python.org<mailto:report at bugs.python.org>>
<http://bugs.python.org/issue2550>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve.com
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org<mailto:Python-Dev at python.org>
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080408/253b6341/attachment.htm 

From bignose+hates-spam at benfinney.id.au  Wed Apr  9 03:37:07 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Wed, 09 Apr 2008 11:37:07 +1000
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are	 good for you
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <87bq4jolks.fsf@benfinney.id.au>

zooko <zooko at zooko.com> writes:

> I am skeptical that prorgammers are going to be willing to use a new
> database format. They already have a database -- their filesystem --
> and they already have the tools to control it -- mv, rm, and
> PYTHONPATH. Many of them already hate the existence the
> "easy_instlal.pth" database file, and I don't see why a new database
> file would be any different.

Moreover, many of us already have a database of *all* packages on the
system, not just Python-language ones: the package database of our
operating system. Adding another, parallel, database which needs
separate maintenance, and only applies to Python packages, is not a
step forward in such a situation.

> They both agreed that it made perfect sense. I told one of them
> about the alternate proposal to define a new database file to
> contain a list of installed packages, and he sighed and rolled his
> eyes and said "So they are planning to reinvent apt!".

That's pretty much my reaction, too.

-- 
 \      "Contentment is a pearl of great price, and whosoever procures |
  `\        it at the expense of ten thousand desires makes a wise and |
_o__)                                   happy purchase."  -- J. Balguy |
Ben Finney


From Scott.Daniels at Acm.Org  Wed Apr  9 05:01:10 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 08 Apr 2008 20:01:10 -0700
Subject: [Python-Dev] py3k: print function treats sep=None and end=None
 in an unintuitive way
In-Reply-To: <200804081801.15687.alessandro.guido@gmail.com>
References: <200804081522.50764.alessandro.guido@gmail.com>	<47FB79B4.5020605@gmail.com>
	<200804081801.15687.alessandro.guido@gmail.com>
Message-ID: <fthb3n$9h7$1@ger.gmane.org>

Alessandro Guido wrote:
> Nick Coghlan wrote:
> Eric Smith wrote:
>> Because None means 'use the default value'.  You probably want:
>> print('a', 'b', sep='', end='')
> 
> I think this is a "not optimally designed" API
> because you have to read the documentation to understand why

Excuse me, I don't know about you, but I don't mind a language
with a document to consult.  I actually wasn't born understanding
_any_ computer (or for that matter natural) language.

0Scott David Daniels
Scott.Daniels at Acm.Org




From tjreedy at udel.edu  Wed Apr  9 05:47:34 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 8 Apr 2008 23:47:34 -0400
Subject: [Python-Dev] unittest's redundant assertions:
	assertsvs.	failIf/Unlesses
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com><ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<47FB33C6.4080008@voidspace.org.uk>
Message-ID: <fthe8i$gpd$1@ger.gmane.org>


"Michael Foord" <fuzzyman at voidspace.org.uk> wrote in message 
news:47FB33C6.4080008 at voidspace.org.uk...
| > Someone please open a bug for this task.
| >
| >
| This sounds like a good compromise and I'm happy to take on the cleanup
| - unless someone else beats me to it. I guess it should wait until 3.0
| final is out of the door before adding the DeprecationWarnings.

I think, however, that the docs should be revised now, for 2.6/3.0.
In the list of methods under TestCase Objects, the preferred method of each 
pair (or triplit) should be given first and the others marked as future 
deprecations, not recommended for new test code.  The explanations of the 
methods should use the preferred names.  So failIf/assertFalse should be 
reversed in order (assuming that failIf is the one to go) and the text "The 
inverse of the failUnless() method" should be changed to "The inverse of 
the assertTrue() method" (if that is the one to be kept).

I would also (lesser priority) revise examples to only use the preferred 
names.  The would make things easiest for new unittest users that are not 
Java refugees.

tjr




From pje at telecommunity.com  Wed Apr  9 06:41:32 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 00:41:32 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
Message-ID: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>

At 10:49 PM 4/8/2008 -0400, Stanley A. Klein wrote:
>On Tue, April 8, 2008 9:37 pm, Ben Finney
><bignose+hates-spam at benfinney.id.au> wrote:
> > Date: Wed, 09 Apr 2008 11:37:07 +1000
> > From: Ben Finney <bignose+hates-spam at benfinney.id.au>
> > Subject: Re: [Distutils] how to easily consume just the parts of eggs
> >       that    are     good for you
> > To: Distutils-Sig at Python.Org
> >
> >
> > zooko <zooko at zooko.com> writes:
> >> eyes and said "So they are planning to reinvent apt!".
> >
> > That's pretty much my reaction, too.
>
>I have the same reaction.

I'm curious.  Have any of you actually read PEP 262 in any detail?  I 
have seen precious little discussion so far that doesn't appear to be 
based on significant misunderstandings of either the purpose of 
reviving the PEP, or the mechanics of its proposed implementation.


>I have tried in the past to use easy_install, but have run into problems
>because there is no communication between easy_install and the rpm
>database, resulting in failure of easy_install to recognize that
>dependencies have already been installed using rpms.

This problem doesn't exist with Python 2.5, unless you're using a 
platform that willfully strips out the installation information that 
Python 2.5 provides for these packages.


>A database focused only on Python packages is highly inappropriate for
>Linux systems, violates the Linux standards, and creates problems because
>eggs are not coordinated with the operating system package manager.

The revamp of PEP 262 is aimed at removing .egg files and directories 
from the process, by allowing system packagers to tell Python what 
files belong to them and should not be messed with.  And conversely, 
allowing systems and installation targets *without* package managers 
to safely manage their Python installations.


>   The
>way to achieve a database for Python would be to provide tools for
>conversion of eggs to rpms and debs,

Such tools already exist, although the conversion takes place from 
source distributions rather than egg distributions.


>to have eggs support conformance to
>the LSB and FHS,

Applying LSB and FHS to the innards of Python packages makes as much 
sense as applying them to the contents of Java .jar files -- i.e., 
none.  If it's unchanging data that's part of a program or library, 
then it's a program or library, just like static data declared in a C 
program or library.  Whether the file extension is .py, .so, or even 
.png is irrelevant.


From fdrake at acm.org  Wed Apr  9 06:56:18 2008
From: fdrake at acm.org (Fred Drake)
Date: Wed, 9 Apr 2008 00:56:18 -0400
Subject: [Python-Dev] unittest's redundant assertions:
	assertsvs.	failIf/Unlesses
In-Reply-To: <fthe8i$gpd$1@ger.gmane.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com><ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<47FB33C6.4080008@voidspace.org.uk> <fthe8i$gpd$1@ger.gmane.org>
Message-ID: <3FC7EDB8-A5A8-48B1-91D0-455DEC90405D@acm.org>

On Apr 8, 2008, at 11:47 PM, Terry Reedy wrote:
> I think, however, that the docs should be revised now, for 2.6/3.0.
> In the list of methods under TestCase Objects, the preferred method  
> of each
> pair (or triplit) should be given first and the others marked as  
> future
> deprecations, not recommended for new test code.


+1 !

This kind of information, once known, should be included in all  
maintained versions of the documentation so that users can build good  
coding habits, or have more time to adjust existing code.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From tnelson at onresolve.com  Wed Apr  9 12:00:52 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 9 Apr 2008 03:00:52 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47F5E6EC.6030402@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk>,<47F4DE23.3020202@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>
	<47F5E6EC.6030402@timgolden.me.uk>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB610@EXMBX04.exchhosting.com>



> -----Original Message-----
> From: Tim Golden [mailto:mail at timgolden.me.uk]
> Sent: 04 April 2008 09:30
> To: Trent Nelson
> Cc: Python-Dev
> Subject: Re: [Python-Dev] fixing tests on windows

> Yes. I'm trying desperately hard to stick to a narrow remit
> of getting tests to run consistently in the face of messy
> file-locking semantics under Windows. I really don't want to
> wade into the minor minefield of making all the tests run
> with consistent temp file semantics. But I may have to.

Hey Tim, any progress on this?  I'd like to start working on this towards the weekend...

        Trent.

From pje at telecommunity.com  Wed Apr  9 15:00:01 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 09:00:01 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409080016.GB1761@phare.normalesup.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<20080409080016.GB1761@phare.normalesup.org>
Message-ID: <20080409125953.99EE63A40D7@sparrow.telecommunity.com>

At 10:00 AM 4/9/2008 +0200, Gael Varoquaux wrote:
>On Wed, Apr 09, 2008 at 12:41:32AM -0400, Phillip J. Eby wrote:
> > >The way to achieve a database for Python would be to provide tools for
> > >conversion of eggs to rpms and debs,
>
> > Such tools already exist, although the conversion takes place from
> > source distributions rather than egg distributions.
>
>What is the status of the deb backend? The only one I know is unofficial
>maintained by Andrew Straw, but my information my be lagging behind.

I was under the impression that there were 2 .deb tools, neither one 
"official" in any sense, any more than 'bdist_rpm' is really 
"official" for RPM-based systems.


>By the way, if these tools work well, they are priceless!

I haven't had need to use any of them, so I don't really know.


From mail at timgolden.me.uk  Wed Apr  9 15:04:57 2008
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 09 Apr 2008 14:04:57 +0100
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB610@EXMBX04.exchhosting.com>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>	<47F33DE7.3030702@timgolden.me.uk>	<47F3592D.3080700@timgolden.me.uk>,
	<47F4DE23.3020202@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>
	<47F5E6EC.6030402@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB610@EXMBX04.exchhosting.com>
Message-ID: <47FCBEF9.9080308@timgolden.me.uk>

Trent Nelson wrote:
> 
>> -----Original Message-----
>> From: Tim Golden [mailto:mail at timgolden.me.uk]
>> Sent: 04 April 2008 09:30
>> To: Trent Nelson
>> Cc: Python-Dev
>> Subject: Re: [Python-Dev] fixing tests on windows
> 
>> Yes. I'm trying desperately hard to stick to a narrow remit
>> of getting tests to run consistently in the face of messy
>> file-locking semantics under Windows. I really don't want to
>> wade into the minor minefield of making all the tests run
>> with consistent temp file semantics. But I may have to.
> 
> Hey Tim, any progress on this?  I'd like to start working on this towards the weekend...

Thanks for the prod, Trent. In short, yes, I spent some time on
this over the weekend but haven't had time since. I found myself 
becoming more and more worried at the amount I had to change, 
especially given rumblings on the list concerning not changing 
the tests if possible.

The possibilities I've been pursuing are these:

1) Leave TESTFN as-is and make the .unlink more robust

2) Leave TESTFN static but instead of "./@TEST" use the
tempfile module to generate a random name in whatever
temp dir applies for the test system. And make the .unlink
more robust.

3) Drop TESTFN and have a function testfn () which generates
a new tempdir filename every time. And make the .unlink more
robust.

Each of these approaches has had its problems, and I'm additionally
hampered by the fact that I don't run *nix or OSX (or any other
platform). I spent far more time than I really had setting up
a VMWare player with Ubuntu but I haven't got it to the point
of building the tests which won't run under Windows.

Pros and cons:

1) Running with a fairly aggressive directory watcher, this doesn't
do enough to make the tests worth. Not worth it.

2) A bit better, especially since the betting is that most of the
suspect apps (local search engines etc.) won't index %TEMP%. But...
certain tests run into problems because they assume that the filename
of TESTFN has no escape-needy characters such as the backslash. Need
to update at least those tests.

3) [This is what I was trying over the weekend]. Loads of work and
quite intrusive. I'll try to get hold of my patch this evening and
mail it over to you. Part of the problem is working out where each
test is expecting TESTFN to be the same file each time and where
it doesn't care.

I was heartened to see that you'd gone ahead with the -- almost as
invasive -- socket test changes. If you'd like to take on what I've
got so far, or simply to adopt your own strategy, please feel free.

TJG

From tnelson at onresolve.com  Wed Apr  9 16:12:58 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 9 Apr 2008 07:12:58 -0700
Subject: [Python-Dev] Next monthly sprint/bugfix day?
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>

Hi,

Is there another online sprint/bugfix day in the pipeline?  If not, can there be? ;-)

        Trent.

From tnelson at onresolve.com  Wed Apr  9 16:44:50 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 9 Apr 2008 07:44:50 -0700
Subject: [Python-Dev] fixing tests on windows
In-Reply-To: <47FCBEF9.9080308@timgolden.me.uk>
References: <ee2a432c0803312342i659753b7g536f24d1989540e0@mail.gmail.com>
	<d11dcfba0804010800v5335a079pa7242a92fb50441b@mail.gmail.com>
	<47F252CE.1020702@timgolden.me.uk>	<47F25F2E.4070408@timgolden.me.uk>
	<52dc1c820804012326y1f8df17ega2955b8c220785ce@mail.gmail.com>
	<47F33DE7.3030702@timgolden.me.uk>
	<47F3592D.3080700@timgolden.me.uk>,<47F4DE23.3020202@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E18FBB27E@EXMBX04.exchhosting.com>
	<47F5E6EC.6030402@timgolden.me.uk>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB610@EXMBX04.exchhosting.com>
	<47FCBEF9.9080308@timgolden.me.uk>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB7A8@EXMBX04.exchhosting.com>


> Thanks for the prod, Trent. In short, yes, I spent some time
> on this over the weekend but haven't had time since. I found
> myself becoming more and more worried at the amount I had to
> change, especially given rumblings on the list concerning not
> changing the tests if possible.

> I was heartened to see that you'd gone ahead with the --
> almost as invasive -- socket test changes. If you'd like to
> take on what I've got so far, or simply to adopt your own
> strategy, please feel free.

I don't hold any weight to rumblings regarding "not changing the tests if possible".  The tests need to change.  They were never written with the intent to run in parallel without any resource contention.  That's an important goal, now.  Please send over any patches you've got, everything will be useful!  Now that the dust has (sort of) settled on the network-oriented tests, I'm ready to start attacking the I/O ones.

        Trent.

From guido at python.org  Wed Apr  9 18:37:50 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Apr 2008 09:37:50 -0700
Subject: [Python-Dev] unittest's redundant assertions: assertsvs.
	failIf/Unlesses
In-Reply-To: <3FC7EDB8-A5A8-48B1-91D0-455DEC90405D@acm.org>
References: <e23fbae90803190024t349e74d5yb895c0f54df27a58@mail.gmail.com>
	<5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com>
	<644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org>
	<dcbbbb410803190721t3390a56chd2ab4794e8d930cb@mail.gmail.com>
	<20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com>
	<5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com>
	<ca471dc20804071737y56520750j83884a2ee5b540f3@mail.gmail.com>
	<47FB33C6.4080008@voidspace.org.uk> <fthe8i$gpd$1@ger.gmane.org>
	<3FC7EDB8-A5A8-48B1-91D0-455DEC90405D@acm.org>
Message-ID: <ca471dc20804090937s16880c3crf0031eaf3e0cb3b8@mail.gmail.com>

On Tue, Apr 8, 2008 at 9:56 PM, Fred Drake <fdrake at acm.org> wrote:
> On Apr 8, 2008, at 11:47 PM, Terry Reedy wrote:
>  > I think, however, that the docs should be revised now, for 2.6/3.0.
>  > In the list of methods under TestCase Objects, the preferred method
>  > of each
>  > pair (or triplit) should be given first and the others marked as
>  > future
>  > deprecations, not recommended for new test code.

>  +1 !
>
>  This kind of information, once known, should be included in all
>  maintained versions of the documentation so that users can build good
>  coding habits, or have more time to adjust existing code.

Great point, Fred! This sounds like a general strategy that we might
employ more often -- when deprecating something, the first step ought
to be to document the best practice.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amcnabb at mcnabbs.org  Wed Apr  9 18:24:22 2008
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Wed, 9 Apr 2008 10:24:22 -0600
Subject: [Python-Dev] how to easily consume just the parts of eggs
	that	are good for you
In-Reply-To: <87bq4jolks.fsf@benfinney.id.au>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<87bq4jolks.fsf@benfinney.id.au>
Message-ID: <20080409162422.GB2442@mcnabbs.org>

On Wed, Apr 09, 2008 at 11:37:07AM +1000, Ben Finney wrote:
> 
> Moreover, many of us already have a database of *all* packages on the
> system, not just Python-language ones: the package database of our
> operating system. Adding another, parallel, database which needs
> separate maintenance, and only applies to Python packages, is not a
> step forward in such a situation.

I agree with you completely.  There are three things I can see myself
wanting from an easy_install command:

1) To download and install an RPM/dpkg to a system-wide location.  The
package is then removable using the normal system package manager.

2) To download the source of a package.

3) To install a package to my home directory.  I would much rather that
this hypothetical easy_install did `rpm --prefix "$HOME/.local" -ivh
some_package.rpm` than have a Python-specific database.

> > They both agreed that it made perfect sense. I told one of them
> > about the alternate proposal to define a new database file to
> > contain a list of installed packages, and he sighed and rolled his
> > eyes and said "So they are planning to reinvent apt!".
> 
> That's pretty much my reaction, too.

Me, too.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20080409/deba6c62/attachment.pgp 

From p.f.moore at gmail.com  Wed Apr  9 19:23:50 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 9 Apr 2008 18:23:50 +0100
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
Message-ID: <79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>

On 09/04/2008, Stanley A. Klein <sklein at cpcug.org> wrote:
> IMHO, the main system without a package manager is Windows.  A reasonable
>  way to deal with Windows would be to create a package manager for it that
>  could be used by Python and anyone else who wanted to use it.  The package
>  manager could establish a file hierarchy similar to the Unix FHS and
>  install files appropriately, except for what is needed to satisfy the
>  Windows OS.  That would probably go a long way to addressing the issues
>  being discussed here.  This is primarily a Windows problem, not a Python
>  problem.

Windows does have a package manager - the add/remove programs
application. It's extremely limited, and doesn't make any attempt at
doing dependency resolution, certainly - but that's a separate issue.

I don't know if you use Windows (as in, develop programs using Python
on Windows). If you do, then I'd be interested in your views on
bdist_wininst and bdist_msi installers, and how they fit into the
setuptools/egg environment, particularly with regard to the package
manager you are proposing. If you don't use Windows, then I don't see
how you can usefully comment.

Personally, as I've said before, I don't have a problem with a
Python-only package manager, as long as it replaces or integrates
bdist_wininst and bdist_msi. Having two package managers is far worse
than having none - and claiming that add/remove programs "isn't a
package manager" is just ignoring reality (if it isn't, then why do
bdist_wininst and bdist_msi exist?).

Are the Linux users happy with having a Python package manager that
ignores RPM/apt? Why should Windows users be any happier?

Sorry - I'm feeling a little grumpy. I've read one too many "Windows
is so broken that people who use it obviously don't care about doing
things right" postings this week :-(

Paul.

From zooko at zooko.com  Wed Apr  9 19:30:55 2008
From: zooko at zooko.com (zooko)
Date: Wed, 9 Apr 2008 10:30:55 -0700
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
Message-ID: <593FD433-F1A0-4445-AA9D-BD7B75DE8125@zooko.com>

On Apr 8, 2008, at 9:41 PM, Phillip J. Eby wrote:
>
> I'm curious.  Have any of you actually read PEP 262 in any detail?

I read it, though not in fine detail.

I didn't write that you are planning to reinvent apt.  I wrote that  
when programmers hear about this PEP they exclaim "They are planning  
to reinvent apt!".

That is a matter of perception and marketing -- the value that I want  
to gain from Python packages is the value of a critical mass of good  
programmers using compatible tools for code re-use.  If a lot of  
programmers hate an idea, then it doesn't matter what the details are  
-- it isn't going to provide this value to me.

I think part of our disagreement is that we are talking about two  
overlapping use cases: programmer and sysadmin (and "end user" which  
is much like sysadmin).  I am not, at this time, interested in the  
sysadmin use case.  As I've mentioned, my sysadmin needs are  
currently well satisfied by apt (and sometimes by GNU stow), and my  
fellow sysadmins with whom I work are absolutely not going to relax  
their "apt-only policy" for the forseeable future, so I cannot use  
such a tool unless it is named "apt" and written by Debian/Ubuntu.

On the other hand I am very interested in the programmer use case,  
because setuptools/easy_install already works pretty well for that,  
and we are already very close to achieving a critical mass of good  
programmers.  Recently several more packages that my project [1]  
relies on have become easy_installable -- Twisted, pywin32 (thanks to  
you, PJE), and foolscap -- and several more have had bugfixes which  
make them work better with easy_install/setuptools -- Nevow and  
zope.interface -- and there are some patches in the queue to make  
another one compatible with setuptools -- pyflakes [2, 3].

So setuptools/easy_install is already (slowly) winning.  I want to  
accelerate that process by reducing the degree to which it is  
incompatible, inconvenient, or objectionable to other programmers.

PEP 262 sounds like a non-starter to me because

1.  It appears to be backwards-incompatible with setuptools/ 
easy_install/eggs, thus losing all the recently gained cooperation  
that I mentioned in the previous paragraph, and

2.  It defines a new database file, where I would prefer either:
    a.  Doing away with database files entirely and relying on the  
filesystem alone to hold that information, or
    b.  Continuing to use the current ".pth" database file format,  
possibly improved by having native support for .pth files in the  
Python import machinery.

3.  Because of #2, it triggers programmers to exclaim "They are  
planning to reinvent apt!", thus making it unlikely that the new  
proposal will recapture the cooperation that setuptools has already  
(slowly) gained.

I'm sorry, PJE -- I know it must be frustrating to you to have your  
proposal criticized on social rather than technical grounds -- but  
social benefits are what I am seeking right now.

Perhaps PEP 262 and my proposal are not actually alternatives, but  
are complementary.  I do not object to Python maintaining a database  
of installed packages for itself (although I cannot *rely* upon such  
behavior, not least because I will be maintaining backwards  
compatibility with Python 2.4 for at least the next several years,  
and with Python 2.5 for at least the next several years after that).  
What I want is for the already implemented, tested, and deployed code- 
re-use features of setuptools/easy_install to be more widely  
accepted.  This is best and most easily achieved by fixing the two  
most frequent objections to setuptools/easy_install: 1.  That you  
can't conveniently install into an arbitrary directory, and 2. that  
it subverts the meaning of your PYTHONPATH.

Regards,

Zooko

[1] http://allmydata.org/source/tahoe/trunk/docs/install.html
[2] http://divmod.org/trac/ticket/2535
[3] http://divmod.org/trac/ticket/2048

From zooko at zooko.com  Wed Apr  9 21:30:17 2008
From: zooko at zooko.com (zooko)
Date: Wed, 9 Apr 2008 12:30:17 -0700
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are good for you
In-Reply-To: <47FC0174.3070605@canterbury.ac.nz>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<47FC0174.3070605@canterbury.ac.nz>
Message-ID: <2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>


On Apr 8, 2008, at 4:36 PM, Greg Ewing wrote:
>
> I discovered another annoyance with eggs the other day -- it
> seems that tracebacks referring to egg-resident files contain the
> pathname of some temporary directory that existed when the egg
> was being packaged, rather than the one it actually exists in
> at run time.

Brian Warner and I discovered that issue yesterday, too.  We  
determined that if you install the egg (with easy_install or with a  
setuptools-powered ./setup.py install) in unzipped form then the  
source file names get rewritten so that your stack traces come with  
source lines.

If you have a package which requires stack traces to come with source  
lines, then you could pass "zip_safe=False" to the call to setup().

I would prefer that zip_safe=False were the default and that either  
the producer or the consumer of a package had to specifically choose  
zip_safe=True in order to install eggs in zipped form.

I've opened a ticket on my setuptools trac:

http://allmydata.org/trac/setuptools/ticket/4

Regards,

Zooko


From pje at telecommunity.com  Wed Apr  9 21:40:53 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 15:40:53 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
Message-ID: <20080409194044.1A4E03A406A@sparrow.telecommunity.com>

At 11:52 AM 4/9/2008 -0400, Stanley A. Klein wrote:
>However, are you implying that the installation information for Python egg
>packages accesses and coordinates with the rpm database?

Yes, when the information isn't stripped out.  Try a more recent Fedora.


>IMHO, the main system without a package manager is Windows.

You're ignoring shared environments and development 
environments.  (Not to mention Mac OS.)


>   A reasonable
>way to deal with Windows would be to create a package manager for it that
>could be used by Python and anyone else who wanted to use it.

Let us know when you've finished it, along with the one for Mac OS.  :)

Of course this still won't do anything for shared environments and 
development environments.


>You are talking here about bdist_rpm and not about a tool that would take
>a Python package distributed as an egg file and convert the egg to an rpm
>or a deb.  Unfortunately, some Python packagers are beginning to limit
>their focus only to egg distribution.  That creates a problem for users
>who have native operating system package management.

That is indeed a problem -- but it's a social one, not a technical 
one.  It's trivial for the publisher of an egg to change their 
command line from "setup.py bdist_egg upload" to "setup.py sdist 
bdist_egg upload", as soon as their users (politely) request that they do so.


> > Applying LSB and FHS to the innards of Python packages makes as much
> > sense as applying them to the contents of Java .jar files -- i.e.,
> > none.  If it's unchanging data that's part of a program or library,
> > then it's a program or library, just like static data declared in a C
> > program or library.  Whether the file extension is .py, .so, or even
> > .png is irrelevant.
>
>The FHS defines places to put specific kinds of files, such as command
>scripts (/bin, /usr/bin, /sbin, or /usr/sbin), documentation
>(/usr/share/doc/package-name), and configuration files (/etc).  There are
>several kinds of files identified and places defined to put them.
>Distribution by eggs has a tendency to scoop up all of those files and put
>them in /usr/lib/python/site-packages, regardless of where they belong.

Eggs don't include documentation or configuration files, and they 
install scripts in script directories, so I don't get what you're 
talking about here.  For any other data that a package accesses at 
runtime, my earlier comments apply.


>Having eggs support conformance to FHS would mean recognizing and tagging
>the relevant files.  A tool for converting eggs to rpms or debs would
>essentially reformat the egg to rpm or deb and put files where they
>belong.

No, because such files as you describe don't exist.  If you think 
they do, then either you have misunderstood the nature of the files 
in question, or the developer has incorrectly placed non-runtime 
files in their installation tree.


From pje at telecommunity.com  Wed Apr  9 21:57:13 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 15:57:13 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
Message-ID: <20080409195704.32E5F3A406A@sparrow.telecommunity.com>

At 10:30 AM 4/9/2008 -0700, zooko wrote:
>PEP 262 sounds like a non-starter to me because
>
>1.  It appears to be backwards-incompatible with setuptools/ 
>easy_install/eggs, thus losing all the recently gained cooperation
>that I mentioned in the previous paragraph, and

No.  It provides a forward-compatibility path for the distutils, so 
that easy_install doesn't have to install things in .egg format in 
the future.  There's no compatibility breakage at all.


>2.  It defines a new database file,

No, it defines *files*.  One file per installed distribution, 
containing (among other things) an installation manifest.


>where I would prefer either:
>    a.  Doing away with database files entirely and relying on the
>filesystem alone to hold that information, or

...which is what PEP 262 *does*.

Unfortunately, PEP 262's title is bad for marketing, as you've 
effectively pointed out.  It would be better titled something 
"package installation manifests" or "package contents files", or 
something of that sort.

>    b.  Continuing to use the current ".pth" database file format,
>possibly improved by having native support for .pth files in the
>Python import machinery.

These mechanisms are orthogonal to this issue.


>3.  Because of #2, it triggers programmers to exclaim "They are
>planning to reinvent apt!", thus making it unlikely that the new
>proposal will recapture the cooperation that setuptools has already
>(slowly) gained.

Yeah, we need a new name.  Everybody is going off of "database of 
installed packages" and thinking "apt", because they aren't paying 
any closer attention.  However, given that we are discussing this on 
Python-Dev and distutils-sig, I do think it's reasonable to expect 
(if perhaps not reasonable to receive) that people discussing the PEP 
have *read* the freaking PEP first, prior to trashing it or offering 
alternatives.

And it's not like I'm personally offended or anything -- I didn't 
even write the PEP in question.  But what's the point of having PEPs 
if people read nothing but their titles?  We could just delete 
everything but PEP 0.  :)


>Perhaps PEP 262 and my proposal are not actually alternatives, but
>are complementary.

As I've already pointed out, your proposal does not address multiple 
installed versions of a package, and I see no sane way to modify it to do so.


>What I want is for the already implemented, tested, and deployed 
>code- re-use features of setuptools/easy_install to be more widely
>accepted.  This is best and most easily achieved by fixing the two
>most frequent objections to setuptools/easy_install: 1.  That you
>can't conveniently install into an arbitrary directory, and 2. that
>it subverts the meaning of your PYTHONPATH.

As I've already stated, the only way for these problems to be fixed 
is for easy_install to not install files in .egg form -- which also 
solves the general objection to using .eggs in the first place.

And the only way to do that, is to have a way to keep track of what 
files are installed.  Rather than have easy_install come up with its 
own way of doing that, I would prefer to share a standard with the 
distutils.  Hence, the PEP discussion.

For earlier versions of Python, it will still be possible to install 
and uninstall with setuptools using this approach.  You just won't be 
able to uninstall pure distutils-based packages, unless you installed 
them using easy_install.

Meanwhile, it has occurred to me that the easiest way of handling 
compatibility is not to require that other packaging tools mark their 
files for non-removability, but simply not allow easy_install to 
remove or overwrite anything that *isn't* claimed by a manifest.  In 
that way, easy_install would be immediately usable in the new mode, 
without any updates to Python or to system packaging tools.


From pje at telecommunity.com  Wed Apr  9 22:06:14 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 16:06:14 -0400
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are good for you
In-Reply-To: <2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<47FC0174.3070605@canterbury.ac.nz>
	<2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>
Message-ID: <20080409200605.DD2783A406A@sparrow.telecommunity.com>

At 12:30 PM 4/9/2008 -0700, zooko wrote:

>On Apr 8, 2008, at 4:36 PM, Greg Ewing wrote:
> >
> > I discovered another annoyance with eggs the other day -- it
> > seems that tracebacks referring to egg-resident files contain the
> > pathname of some temporary directory that existed when the egg
> > was being packaged, rather than the one it actually exists in
> > at run time.
>
>Brian Warner and I discovered that issue yesterday, too.  We
>determined that if you install the egg (with easy_install or with a
>setuptools-powered ./setup.py install) in unzipped form then the
>source file names get rewritten so that your stack traces come with
>source lines.

Are you using Python 2.5?  As of 2.5, the linecache module should 
correctly read the source line from the present location of the 
source file the module was loaded from, regardless of the file name 
specified in the traceback.


>If you have a package which requires stack traces to come with source
>lines, then you could pass "zip_safe=False" to the call to setup().
>
>I would prefer that zip_safe=False were the default and that either
>the producer or the consumer of a package had to specifically choose
>zip_safe=True in order to install eggs in zipped form.

A better fix would be for Python to use relative paths in 
co_filename, as this would fix similar problems that occur whenever 
you move .pyc/.pyo files around.


From sergiodj at linux.vnet.ibm.com  Wed Apr  9 22:44:14 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Wed, 09 Apr 2008 17:44:14 -0300
Subject: [Python-Dev] Compiling Python on a biarch (PPC/PPC64)
Message-ID: <1207773854.5788.23.camel@miki>

Hi everybody :-)

I'm running a PPC64 biarch here (i.e., a 64-bit kernel with 32-bit
userspace), and trying to compile Python on it. Well, as I have a GCC
compiler that generates natively 32-bit executables, everything goes
well when I *don't* use any specific 64-bit flag (./configure
--enable-shared && make && make test && make install).

My problem happens when I try to compile Python for PPC64 using some
configure flags and environment variables, like --target and CFLAGS. To
be more specific, that's what I tried:

./configure --enable-shared --target=powerpc64-unknown-linux
CFLAGS='-m64'

The configure runs fine. But when I do a "make":

sergio at elm3b188 ~/work/w/Python-2.5.2 $ make
gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes  -I. -IInclude -I./Include  -fPIC -DPy_BUILD_CORE -o
Modules/python.o ./Modules/python.c
In file included from Include/Python.h:57,
                 from ./Modules/python.c:3:
Include/pyport.h:761:2: error: #error "LONG_BIT definition appears wrong
for platform (bad gcc/glibc config?)."
make: *** [Modules/python.o] Error 1

Well, now starts the part where I try to understand things, so please
correct me if I'm wrong ;-). After some investigation, I thought that
the error was the (absence of) detection of the building target, so I
wrote a little piece of code in configure.in which detects PPC64 in the
$target variable and appends to $BASECFLAGS the string '-m64'. It seems
to be more than that, though (because that approach didn't work
completely)... Can anyone give me a hand on this? As far as I could
investigate, I must change setup.py in order to get this issue fixed,
right?

Thanks in advance,

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From jimjjewett at gmail.com  Wed Apr  9 23:15:16 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 9 Apr 2008 17:15:16 -0400
Subject: [Python-Dev] windows (was: how to easily consume just the parts of
	eggs that are good for you)
Message-ID: <fb6fbf560804091415l3ecce6d8p354fdeee2caaba0c@mail.gmail.com>

> Are the Linux users happy with having a Python
> package manager that ignores RPM/apt? Why
> should Windows users be any happier?

Because, as you noted, the add/remove programs application is severely
limited.

> I've read one too many "Windows is so broken
> that people who use it obviously don't care about
> doing things right" postings this week

I'm honestly not sure that such fine-grained control is the right user
interface, particularly for a non-shared system.

But even if it were, Windows doesn't really have it, and it isn't so
valuable that a solution which works only for python could do much
better than the existing 3rd-party setup tools.



As a windows user, I don't want python packages showing up in the
add/remove programs list, because it won't help me, and will make the
few times I do use that tool even more awkward.

That said, I agree that if python does package management, offering
windows users the choice of using that application is probably a good
idea.  The catch is that "package managers" seem to offer far more
fine-grained power (even without dependency resolution) than windows.
Duplicating this would add lots of complexity just for windows -- and
still might not be all that useful.  I'm already used to looking for
an uninstall.exe in the  directory of anything I can actually
uninstall, and accepting that most things just don't go away cleanly.
As a programmer, this feels wrong, but ... it is probably a good
tradeoff for the time I don't want to spend maintaining things.

If I really wanted a fancy tool that took care of dependencies and
alternate versions, I would be willing to run something
python-specific, or to treat each package as a subcomponent that I
managed through "Change an existing program" applied to python.

But realistically, I don't see such a tool being used often enough to
justify inclusion in the core.

-jJ

From pje at telecommunity.com  Wed Apr  9 23:25:31 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 17:25:31 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
Message-ID: <20080409212522.9CDE13A409E@sparrow.telecommunity.com>

At 04:43 PM 4/9/2008 -0400, Stanley A. Klein wrote:
>I don't understand what you mean by "shared environments and development
>  environments".

I mean that in a shared or development environment, a system packager 
isn't useful, since it expects things to live in *one* place, and 
usually to have only one *version*, as well.


>I agree that we are dealing with a combination of technical and social
>issues here.  However, I think it takes a lot more understanding for a
>publisher to get everything straight.

If they provide you with the source distribution, you can make any 
sort of package you want.


> > Eggs don't include documentation or configuration files, and they
> > install scripts in script directories, so I don't get what you're
> > talking about here.  For any other data that a package accesses at
> > runtime, my earlier comments apply.
> >
>
>But rpms and debs do include these files, plus manual pages, localization
>files and a lot of other ancillary stuff.

...just one of the many reasons that eggs are not a replacement for 
rpms and debs.  :)


>Most of the Python tarballs I have downloaded have all kinds of files in
>their installation trees.  This is a major pain in the you-know-what for
>someone trying to use bdist_rpm and get proper, FHS-compliant rpms.  If
>eggs are supposed to be strictly runtime files, I think very few
>developers actually understand that.  Better yet, how do you define what
>should be included in an installation?  It sounds like the egg concept
>doesn't include several kinds of files that rpm and deb would include in
>an installation.  I think that may be an important issue here.

It would be, if .eggs were a packaging format, rather than a binary 
distribution/runtime format.

Remember "eggs are to Python as jars are to Java" -- a Java .jar 
doesn't contain documentation either, unless it's needed at 
runtime.  Same for configuration files.

They're not system packages, in other words.  The assumption that 
they are is another marketing failure, due to conflation of "package 
== distribution of python code" and "package == thing you manage with 
a system packager".  People see, "I put my package in an .egg" and 
think it's the latter definition, when it's barely even the former.  :)


From martin at v.loewis.de  Wed Apr  9 23:27:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 09 Apr 2008 23:27:49 +0200
Subject: [Python-Dev] Compiling Python on a biarch (PPC/PPC64)
In-Reply-To: <1207773854.5788.23.camel@miki>
References: <1207773854.5788.23.camel@miki>
Message-ID: <47FD34D5.2060607@v.loewis.de>

> Can anyone give me a hand on this? As far as I could
> investigate, I must change setup.py in order to get this issue fixed,
> right?

Wrong. I didn't read your entire message, but this is definitely wrong.
setup.py isn't invoked until python itself is built, and
Modules/python.c is part of python itself.

Regards,
Martin

From zooko at zooko.com  Thu Apr 10 00:11:01 2008
From: zooko at zooko.com (zooko)
Date: Wed, 9 Apr 2008 15:11:01 -0700
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409125953.99EE63A40D7@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<20080409080016.GB1761@phare.normalesup.org>
	<20080409125953.99EE63A40D7@sparrow.telecommunity.com>
Message-ID: <943A6CC1-6A89-4761-AB7A-332DCDCC7F81@zooko.com>


On Apr 9, 2008, at 6:00 AM, Phillip J. Eby wrote:
>>
>> By the way, if these tools work well, they are priceless!
>
> I haven't had need to use any of them, so I don't really know.

They are easydeb [1] and stddeb [2].  The former appears to be  
incomplete and unmaintained.  The latter appears to be usable, but  
somewhat incomplete -- substantial manual labor is required to use it  
successfully, as documented by my programming partner Brian Warner in  
this ticket: [3].

Regards,

Zooko

[1] http://easy-deb.sourceforge.net/
[2] http://stdeb.python-hosting.com/
[3] http://allmydata.org/trac/tahoe/ticket/251

From zooko at zooko.com  Thu Apr 10 00:20:30 2008
From: zooko at zooko.com (zooko)
Date: Wed, 9 Apr 2008 15:20:30 -0700
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409194044.1A4E03A406A@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
Message-ID: <390014FF-5088-42C1-8213-17D0B25D2FFB@zooko.com>

On Apr 9, 2008, at 12:40 PM, Phillip J. Eby wrote:
>>
>> You are talking here about bdist_rpm and not about a tool that  
>> would take
>> a Python package distributed as an egg file and convert the egg to  
>> an rpm
>> or a deb.  Unfortunately, some Python packagers are beginning to  
>> limit
>> their focus only to egg distribution.  That creates a problem for  
>> users
>> who have native operating system package management.
>
> That is indeed a problem -- but it's a social one, not a technical
> one.  It's trivial for the publisher of an egg to change their
> command line from "setup.py bdist_egg upload" to "setup.py sdist
> bdist_egg upload", as soon as their users (politely) request that  
> they do so.

In general, it would be good if eggs came with .py files by default  
instead of .pyc files.

I've opened a ticket on my setuptools trac about this proposal:

http://allmydata.org/trac/setuptools/ticket/5 # binary eggs should  
come with .py files by default, rather than .pyc files

Regards,

Zooko

From p.f.moore at gmail.com  Thu Apr 10 00:46:19 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 9 Apr 2008 23:46:19 +0100
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
	<36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
Message-ID: <79990c6b0804091546p5cf67f2ta800c675e7baa256@mail.gmail.com>

On 09/04/2008, Stanley A. Klein <sklein at cpcug.org> wrote:
>  I think you raise an interesting issue:  What is a package manager?

My (very simplistic) answer is that it's whatever someone uses to
manage packages. What level of functionality it has is irrelevant, as
long as it suits an individual's way of working.

>  I don't think I ever said that Windows is broken in the area of package
>  management.

I hope I didn't say you had - but it is an often-raised point, and it
does grate on me (as by implication, it says that what I do isn't
"real" package management). It's just another flavour of the old
Windows vs Linux OS wars, which I don't want to participate in :-)

>  My own experience is that the files of Windows programs tend
>  to be put in a directory devoted to the program, rather than put in
>  directories with other files having similar purposes.  At one time, the
>  default location in Windows for word processing files was even in a
>  sub-directory of the word processing program.  That changed to having a
>  form of user home directory, but it didn't change much for the program
>  files themselves.  Unix/Linux puts the files in specific areas of the file
>  system having functional commonality.  One could almost say that the
>  Windows default approach to structuring its filesystem avoids or minimizes
>  the need for package management.

Agreed. The minimal package manager on Windows is arguably reasonable
precisely because the standard layout doesn't require much more.

On the other hand, Microsoft has a bad habit of changing their
"standards", and in particular Vista changes a lot of things. But
that's very off-topic, so I'll avoid going into detail here.

>  I repeat that this issue mainly arises because Windows doesn't have the
>  same kind of filesystem structure (and therefore the need for package
>  management) that other systems have.  I don't know what Windows add/remove
>  programs function does, but all it might do is to run the executable to
>  install packages and record the installation (as was previously done by
>  third party programs) to facilitate clean removal.

Precisely. I could argue that the Linix filesystem structure is over
complex, and causes the need for complex package managers, precisely
because it's impossible to manually keep track of what file is owned
by what package. But this way lies OS wars, so I won't make a major
point of this, but just ask that people consider it as a possibility.

(I believe that Mac OS X goes for an even simpler structure -
applications store *everything* in the one directory, so that
install/uninstall is simply a directory copy/remove. I won't comment
on what that proves...)

>  Unless you can perform
>  more of the other functions I listed above, I doubt I would call
>  add/remove a package manager.

OK. I would, though, as I don't really want much more.

Later, you said:

> I just don't do Windows.  :-)

And you've been pretty clear that you're looking for information
rather than trying to explain how you think Windows should work. But
many people aren't so balanced in their views, and that makes it hard
to have a discussion - I'd suggest that people without Windows
experience leave the Windows side to the Windows people, and accept
that when they say "XXX won't work for us", that the statement is
probably true.

I find this whole discussion hugely confusing, because a lot of people
are stating opinions about environments which it seems they don't use,
or know much about. I don't know how to avoid this, but it does make
it highly unlikely that any practical progress will get made.

Paul.

From p.f.moore at gmail.com  Thu Apr 10 00:48:57 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 9 Apr 2008 23:48:57 +0100
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409212522.9CDE13A409E@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
	<20080409212522.9CDE13A409E@sparrow.telecommunity.com>
Message-ID: <79990c6b0804091548p533e4dear5fb500341e4b87af@mail.gmail.com>

On 09/04/2008, Phillip J. Eby <pje at telecommunity.com> wrote:
> It would be, if .eggs were a packaging format, rather than a binary
>  distribution/runtime format.
>
>  Remember "eggs are to Python as jars are to Java" -- a Java .jar
>  doesn't contain documentation either, unless it's needed at
>  runtime.  Same for configuration files.

And yet, Java doesn't have an equivalent of easy_install for jar
files, to my knowledge. If Python had eggs but no easy_install, maybe
this whole discussion wouldn't be taking place.

(I know I personally like the idea of egg-as-jar-file, but *hate* the
idea of egg-as-dependency-handling-tool-and-everything-else).

Paul.

From bignose+hates-spam at benfinney.id.au  Thu Apr 10 00:47:06 2008
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Thu, 10 Apr 2008 08:47:06 +1000
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are good for you
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
Message-ID: <87iqyqir2t.fsf@benfinney.id.au>

"Stanley A. Klein" <sklein at cpcug.org> writes:

> IMHO, the main system without a package manager is Windows.

AFAICT the MacOS platform also lacks in this area.

> A reasonable way to deal with Windows would be to create a package
> manager for it that could be used by Python and anyone else who
> wanted to use it. [...] This is primarily a Windows problem, not a
> Python problem.

I'd rephrase this as: If you *must* re-invent package management for
those legacy systems without it, please *don't* make it specific to
Python.

-- 
 \      ?The right to search for truth implies also a duty; one must |
  `\    not conceal any part of what one has recognized to be true.? |
_o__)                                               ?Albert Einstein |
Ben Finney


From p.f.moore at gmail.com  Thu Apr 10 00:52:08 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 9 Apr 2008 23:52:08 +0100
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <87iqyqir2t.fsf@benfinney.id.au>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
Message-ID: <79990c6b0804091552h4ee7f9f6t5512ed5f5aa70d2a@mail.gmail.com>

On 09/04/2008, Ben Finney <bignose+hates-spam at benfinney.id.au> wrote:
> "Stanley A. Klein" <sklein at cpcug.org> writes:
>  > A reasonable way to deal with Windows would be to create a package
>  > manager for it that could be used by Python and anyone else who
> > wanted to use it. [...] This is primarily a Windows problem, not a
>  > Python problem.
>
>  I'd rephrase this as: If you *must* re-invent package management for
>  those legacy systems without it, please *don't* make it specific to
>  Python.

And I would say that Windows doesn't have a problem. Are any Windows
users proposing building a package management system for Windows
(Python-specific or otherwise)? It's a genuine question - is this
something that Windows users are after, or is it just Linux users
trying to show Windows users what they are missing?

(BTW, it's unreasonable to call Windows "legacy" - whatever that word
means, Windows ain't it (yet...))

Paul.

From pje at telecommunity.com  Thu Apr 10 01:12:55 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 19:12:55 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <390014FF-5088-42C1-8213-17D0B25D2FFB@zooko.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<390014FF-5088-42C1-8213-17D0B25D2FFB@zooko.com>
Message-ID: <20080409231247.7F3A23A406A@sparrow.telecommunity.com>

At 03:20 PM 4/9/2008 -0700, zooko wrote:
>I've opened a ticket on my setuptools trac about this proposal:
>
>http://allmydata.org/trac/setuptools/ticket/5 # binary eggs should
>come with .py files by default, rather than .pyc files

Filling your tracker with already-rejected proposals isn't likely to 
encourage me to look at it, especially when I've personally rejected 
them to you in IRC.  That goes for your ticket #4 as well.


From pje at telecommunity.com  Thu Apr 10 01:16:20 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 19:16:20 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <79990c6b0804091548p533e4dear5fb500341e4b87af@mail.gmail.co
 m>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
	<20080409212522.9CDE13A409E@sparrow.telecommunity.com>
	<79990c6b0804091548p533e4dear5fb500341e4b87af@mail.gmail.com>
Message-ID: <20080409231611.646C53A406A@sparrow.telecommunity.com>

At 11:48 PM 4/9/2008 +0100, Paul Moore wrote:
>On 09/04/2008, Phillip J. Eby <pje at telecommunity.com> wrote:
> > It would be, if .eggs were a packaging format, rather than a binary
> >  distribution/runtime format.
> >
> >  Remember "eggs are to Python as jars are to Java" -- a Java .jar
> >  doesn't contain documentation either, unless it's needed at
> >  runtime.  Same for configuration files.
>
>And yet, Java doesn't have an equivalent of easy_install for jar
>files, to my knowledge.

Actually, OSGi and Eclipse plugins and "feature sites" come quite 
close, and setuptools rips off many of its features from them.  OSGi 
is basically a standard for additional .jar metadata to encompass 
dependencies and other info.


From dpeterson at enthought.com  Thu Apr 10 01:17:49 2008
From: dpeterson at enthought.com (Dave Peterson)
Date: Wed, 09 Apr 2008 18:17:49 -0500
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409194044.1A4E03A406A@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
Message-ID: <47FD4E9D.8060600@enthought.com>

Phillip J. Eby wrote:
>
>>> Applying LSB and FHS to the innards of Python packages makes as much
>>> sense as applying them to the contents of Java .jar files -- i.e.,
>>> none.  If it's unchanging data that's part of a program or library,
>>> then it's a program or library, just like static data declared in a C
>>> program or library.  Whether the file extension is .py, .so, or even
>>> .png is irrelevant.
>>>       
>> The FHS defines places to put specific kinds of files, such as command
>> scripts (/bin, /usr/bin, /sbin, or /usr/sbin), documentation
>> (/usr/share/doc/package-name), and configuration files (/etc).  There are
>> several kinds of files identified and places defined to put them.
>> Distribution by eggs has a tendency to scoop up all of those files and put
>> them in /usr/lib/python/site-packages, regardless of where they belong.
>>     
>
> Eggs don't include documentation or configuration files, and they 
> install scripts in script directories, so I don't get what you're 
> talking about here.  For any other data that a package accesses at 
> runtime, my earlier comments apply.
>   

We've talked a bit about this before, and IIRC, at that time you 
(Phillip) were willing to consider patches to setuptools that allowed 
for the inclusion of documentation in eggs such that it was placed into 
an LSB/FHS appropriate directory (or some standard dir for non-LSB 
systems) during the install process.   I'm assuming that something 
similar for config files wouldn't be a problem either?  Or is this whole 
idea out the window given the way the discussion has trended and the 
reiteration above that eggs are meant to be similar in principal to jars?  


Not that I have a patch yet, but we've been working on it in our "spare 
time" over here at Enthought.  I'm now wondering if we're wasting our 
time. :-)


I think the biggest use-case confusion in the current discussion is 
whether we're talking about applications or libraries?  If we're talking 
about libraries, then clearly distribution of only executables is 
sufficient because anything else should be handled by the application 
distribution when that library is used in an app.  Whereas if we're 
talking about applications, you probably *do* want to include 
documentation, configuration info, etc. in your distribution.   I think 
I can sum up any further points by simply asking: "Should it be safe to 
assume I can distribute my application via eggs / easy_install just 
because it is written in Python?"


-- Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080409/95acebba/attachment-0001.htm 

From pje at telecommunity.com  Thu Apr 10 01:24:46 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 09 Apr 2008 19:24:46 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that	are good for you
In-Reply-To: <20080409225138.GH28763@phare.normalesup.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
	<36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
	<79990c6b0804091546p5cf67f2ta800c675e7baa256@mail.gmail.com>
	<20080409225138.GH28763@phare.normalesup.org>
Message-ID: <20080409232437.7D74F3A406A@sparrow.telecommunity.com>

At 12:51 AM 4/10/2008 +0200, Gael Varoquaux wrote:
>On Wed, Apr 09, 2008 at 11:46:19PM +0100, Paul Moore wrote:
> > I find this whole discussion hugely confusing, because a lot of people
> > are stating opinions about environments which it seems they don't use,
> > or know much about. I don't know how to avoid this, but it does make
> > it highly unlikely that any practical progress will get made.
>
>I find that something that doesn't help at all the discussion move
>forward is that everybody has different usecases in mind, on different
>platforms, and is not interested in other people's usecases.
>
>Hopefuly I am wrong,

You're not wrong at all.  I have to deal with *all* the platforms and 
use cases, which makes it quite frustrating when people who haven't 
even read the requirements are making proposals to "solve" things in 
ways that break for everyone except their own niche platform+usecase 
combination.

Guido, can I borrow the time machine and go back and NOT try to 
improve on the distutils?  Or is there already too much collateral 
damage to the timeline? ;-)


From greg.ewing at canterbury.ac.nz  Thu Apr 10 01:27:50 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 10 Apr 2008 11:27:50 +1200
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are good for you
In-Reply-To: <2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<47FC0174.3070605@canterbury.ac.nz>
	<2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>
Message-ID: <47FD50F6.1070109@canterbury.ac.nz>

zooko wrote:
> 
> We  determined 
> that if you install the egg (with easy_install or with a  
> setuptools-powered ./setup.py install) in unzipped form then the  source 
> file names get rewritten so that your stack traces come with  source lines.

That wouldn't have helped me with my problem, because I was
trying to use the traceback to track down where the file was
in the first place. I wasn't even aware that it was in an
egg at first.

I wouldn't have been using eggs at all if I hadn't been
tricked into it by some package telling me I had to use
easy_install to install it. If I'd known that setuptools
would make intrusive changes to the behaviour of my
system, I would never have touched it.

Is there a way of *un*installing setuptools? I'd like to
put my Python back the way it used to be.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Apr 10 01:58:53 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 10 Apr 2008 11:58:53 +1200
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are good for you
In-Reply-To: <20080409200605.DD2783A406A@sparrow.telecommunity.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<47FC0174.3070605@canterbury.ac.nz>
	<2869C14C-A204-4A71-9C5E-8E202E1DC42D@zooko.com>
	<20080409200605.DD2783A406A@sparrow.telecommunity.com>
Message-ID: <47FD583D.30900@canterbury.ac.nz>

Phillip J. Eby wrote:
> Are you using Python 2.5?  As of 2.5, the linecache module should 
> correctly read the source line from the present location of the source 
> file the module was loaded from, regardless of the file name specified 
> in the traceback.

I think it was doing that, but I was trying to find the file,
and the traceback wasn't helping.

> A better fix would be for Python to use relative paths in co_filename, 
> as this would fix similar problems that occur whenever you move 
> .pyc/.pyo files around.

Yes, it must be possible to do something better. The actual path
is known when the module is loaded, so it should just be a matter
of storing it somewhere appropriate.

-- 
Greg


From zooko at zooko.com  Thu Apr 10 02:27:22 2008
From: zooko at zooko.com (zooko)
Date: Wed, 9 Apr 2008 17:27:22 -0700
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409231247.7F3A23A406A@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<390014FF-5088-42C1-8213-17D0B25D2FFB@zooko.com>
	<20080409231247.7F3A23A406A@sparrow.telecommunity.com>
Message-ID: <09411577-DD0F-4E0C-AE69-6CE4F1D97CD5@zooko.com>


On Apr 9, 2008, at 4:12 PM, Phillip J. Eby wrote:

>> http://allmydata.org/trac/setuptools/ticket/5 # binary eggs should
>> come with .py files by default, rather than .pyc files
>
> Filling your tracker with already-rejected proposals isn't likely  
> to encourage me to look at it, especially when I've personally  
> rejected them to you in IRC.  That goes for your ticket #4 as well.

Part of my motivation in maintaining this tracker is to take issue  
discussions from IRC, and from mailing lists, and make them more  
permanent and structured.  This part is useful even for rejected  
proposals, as an historical record that other people interested in  
those issues can consult.

I will mark those two tickets as "rejected by PJE".  Could you please  
repeat (so that I don't misrepresent you due to my faulty memory of  
our IRC discussion from more than a year ago) your reason for  
rejecting these two:

http://allmydata.org/trac/setuptools/ticket/4 (when considering  
whether to zip, err on the side of safety rather than performance)

http://allmydata.org/trac/setuptools/ticket/5 (binary eggs should  
come with .py files by default, rather than .pyc files)

You are of course welcome to log in to that trac and update those  
tickets yourself, but if you prefer not to then I will paste your  
reasons into those tickets.

Regards,

Zooko

From lxander.m at gmail.com  Thu Apr 10 03:54:05 2008
From: lxander.m at gmail.com (Alexander Michael)
Date: Wed, 9 Apr 2008 21:54:05 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <20080409194044.1A4E03A406A@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
Message-ID: <525f23e80804091854g19460b02pd426016b56177272@mail.gmail.com>

On Wed, Apr 9, 2008 at 3:40 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
>  That is indeed a problem -- but it's a social one, not a technical
>  one.  It's trivial for the publisher of an egg to change their
>  command line from "setup.py bdist_egg upload" to "setup.py sdist
>  bdist_egg upload", as soon as their users (politely) request that they do so.

I know you say it in
<http://peak.telecommunity.com/DevCenter/setuptools#distributing-a-setuptools-based-project>,
but perhaps you should be more explicit about the best practice being
to distribute an sdist as well as one or more eggs and highlight this
with some standout text like:

"""
When uploading your project to PyPI, always upload your sdist in
addition to any eggs unless you have a good idiosyncratic reason not
to. The easy_install tool can also download and install sdist's on all
platforms (universally for pure-python packages and provided compilers
are available for distributions containing python extensions), so
uploading them gives your project the widest possible audience.
"""

Perhaps this will help some the social challenges.

Regards,
Alex

From stephen at xemacs.org  Thu Apr 10 06:12:40 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 10 Apr 2008 13:12:40 +0900
Subject: [Python-Dev] how to easily consume just the parts of eggs
	that	are good for you
In-Reply-To: <87iqyqir2t.fsf@benfinney.id.au>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
Message-ID: <87ej9eic07.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:
 > "Stanley A. Klein" <sklein at cpcug.org> writes:
 > 
 > > IMHO, the main system without a package manager is Windows.
 > 
 > AFAICT the MacOS platform also lacks in this area.

Actually, they both have them.  Windows has Cygwin (rpm-based), while
for MacOS Fink (deb-based), MacPorts (FreeBSD ports-like), and
NetBSD's pkgsrc are all viable options if you want packaging support
for 3rd-party packages.


From steven.bethard at gmail.com  Thu Apr 10 08:11:22 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 10 Apr 2008 00:11:22 -0600
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that are good for you
In-Reply-To: <79990c6b0804091548p533e4dear5fb500341e4b87af@mail.gmail.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
	<12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
	<20080409212522.9CDE13A409E@sparrow.telecommunity.com>
	<79990c6b0804091548p533e4dear5fb500341e4b87af@mail.gmail.com>
Message-ID: <d11dcfba0804092311h10e5d227l27447f279b006c8d@mail.gmail.com>

On Wed, Apr 9, 2008 at 4:48 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 09/04/2008, Phillip J. Eby <pje at telecommunity.com> wrote:
>  > It would be, if .eggs were a packaging format, rather than a binary
>  >  distribution/runtime format.
>  >
>  >  Remember "eggs are to Python as jars are to Java" -- a Java .jar
>  >  doesn't contain documentation either, unless it's needed at
>  >  runtime.  Same for configuration files.
>
>  And yet, Java doesn't have an equivalent of easy_install for jar
>  files, to my knowledge. If Python had eggs but no easy_install, maybe
>  this whole discussion wouldn't be taking place.
>
>  (I know I personally like the idea of egg-as-jar-file, but *hate* the
>  idea of egg-as-dependency-handling-tool-and-everything-else).

So then do you really need anything more than the 2.6 support for
executing directories (and zipfiles) with a __main__ file?

    $ python_d.exe --version
    Python 2.6a1+

    $ more foo\__main__.py
    print 'here I am!'

    $ python_d.exe foo
    here I am!
    [8698 refs]

    $ python_d.exe foo.zip
    here I am!
    [8563 refs]

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From apt.shansen at gmail.com  Thu Apr 10 09:12:43 2008
From: apt.shansen at gmail.com (Stephen Hansen)
Date: Thu, 10 Apr 2008 00:12:43 -0700
Subject: [Python-Dev] how to easily consume just the parts of eggs that
	are good for you
In-Reply-To: <87ej9eic07.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
	<87ej9eic07.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <7a9c25c20804100012y1a9d797cv1ed4c6d0739c5ba0@mail.gmail.com>

>
>  > > IMHO, the main system without a package manager is Windows.
>  >
>  > AFAICT the MacOS platform also lacks in this area.
>
> Actually, they both have them.  Windows has Cygwin (rpm-based), while
> for MacOS Fink (deb-based), MacPorts (FreeBSD ports-like), and
> NetBSD's pkgsrc are all viable options if you want packaging support
> for 3rd-party packages.
>

Er, excuse me for cutting in but-- that's just not at all the same thing.

For people who are using a Red Had derivative, or a Debian derivative, or ..
whatever .. the package manager isn't something they go out of their way to
add and then have to struggle with. It's simply *how* their world works. By
a large measure, everything they want is there, managed by their package
management system.

For those users, I understand well that they don't want some Python package
management system to be a second system that they have to deal with.

But I'm sorry: the world is bigger then Linux and such things.

I'm a mac user, who has had extensive experience in Linux; but on the mac?
Fink and MacPorts added on top of MacOSX is not even remotely comparable to
using an operating system which has a standard package manager that is a
part of every users daily life. My operating system is Unixy, and comes
pre-installed with a number of things, including Python, wxWidgets, sqlite--
many things that make the programs I make for my customers easy to use.

But for those products that are *not* available standard, where are my mac
customers left? Their options are to install something like Fink, or
MacPorts... and then we come into issues of it wanting to install its *own*
version of python, or its *own* version of these third party things, on top
of what's already there? The alternative is that users have to install,
manually, these third-party requirements themselves.

I've found that it is in general far easier for me to just download and
install stuff manually then to rely on these "Add-on" package managers. At
least if I'm thinking of providing as minimally and least intrusive as
possible experience for my users to install my product.

Power users, especially those familiar with the Linux world, may relish in
the existence of MacPorts and Fink... Regular people, even IT managers of
companies I have to deal with-- will not.

I love easy_install/setuptools because it lets me get my *Python*
applications and products out to people, regardless of OS, in a way that
just *works*.

I do think its valuable to do so in a way that will integrate with native
package managers on those operating systems that they are a native and
integrated part of -- but to say, "Let's not re-create apt!" is a sorrowful
stance. It's saying, "Screw Windows, because it isn't as good as what we
have." and "Screw Mac, because its not as good as we have." Or even, "Screw
the people who aren't power users and are just not going to be able to go
through the effort of adding *on* a non-standard package management system
to their operating system."

There's a whole wide world out there that simply does *not* have a
"package management"(*) system.

Python is beautiful, making Python programs is blissful. I'd be far, far
more concerned with making it easy to distribute Python-based programs to
*any* operating system then I would be concerned with partially redoing what
a *minority* of systems out there have done to make package management (with
dependencies and all) easy for its users.

Python is a cross-platform development environment. Let's not forget that
most people just... don't have Linux... and don't have the equally blissful
world of apt or rpm available to them natively.

Its very cool to *integrate* -- to make a way for those RPM and
DEB distributors to deliver an app in their own way that will satisfy their
needs. But what about the people *without* that native capability? Having a
Python-only distribution/management system like easy_install is a *huge*
boon to getting real products to real people.

I think PJE's idea here is very good. Just include certain files and such in
the RPM/DEB that will satisfy the "python-package-management" system. For
RPM/DEB users and their OS's database of packages, its irrelevant largely--
they'll still keep using their own system. But if a product needs something
without a .deb or .rpm, or if someone's on an operating system without a
native system-- they can still gather everything they need.

Anyways.

My 2 cents.

--Stephen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080410/be208ce6/attachment-0001.htm 

From him at online.de  Thu Apr 10 09:58:27 2008
From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=)
Date: Thu, 10 Apr 2008 09:58:27 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409212522.9CDE13A409E@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>	<12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>
	<20080409212522.9CDE13A409E@sparrow.telecommunity.com>
Message-ID: <47FDC8A3.7090300@online.de>

Phillip J. Eby wrote:
> It would be, if .eggs were a packaging format, rather than a binary 
> distribution/runtime format.
>
> Remember "eggs are to Python as jars are to Java" -- a Java .jar 
> doesn't contain documentation either, unless it's needed at 
> runtime.  Same for configuration files.
>   
But there's generally no need to easily have a look into a .class file 
with a tool the user
is used to whereas for python, we're often interested in knowing the 
details. And having
a zip file in my way to the source has left me frustrated often enough.

If you want to be consequent and consistent leave out the .py files from 
eggs, a jar file
normally doesn't contain .java sources files either.

> They're not system packages, in other words.  The assumption that 
> they are is another marketing failure, due to conflation of "package 
> == distribution of python code" and "package == thing you manage with 
> a system packager".  People see, "I put my package in an .egg" and 
> think it's the latter definition, when it's barely even the former.  :)
>   
I agree that they are not system packages, but I would have prefered to 
install multiple versions
of a package to separate "site-packages" directories, something that is 
really well understood by
most unsofisticated python programmers. The selection of the version 
could then be made at
runtime by a PYTHONPATH setting and not by fiddling with .pth files 
(something that could
be autmated by a tool and persisted in batch files).


From stephen at xemacs.org  Thu Apr 10 10:30:41 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 10 Apr 2008 17:30:41 +0900
Subject: [Python-Dev] how to easily consume just the parts of eggs
	that	are good for you
In-Reply-To: <7a9c25c20804100012y1a9d797cv1ed4c6d0739c5ba0@mail.gmail.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
	<87ej9eic07.fsf@uwakimon.sk.tsukuba.ac.jp>
	<7a9c25c20804100012y1a9d797cv1ed4c6d0739c5ba0@mail.gmail.com>
Message-ID: <87zls2glhq.fsf@uwakimon.sk.tsukuba.ac.jp>

Stephen Hansen writes:
 > >
 > >  > > IMHO, the main system without a package manager is Windows.
 > >  >
 > >  > AFAICT the MacOS platform also lacks in this area.
 > >
 > > Actually, they both have them.  Windows has Cygwin (rpm-based), while
 > > for MacOS Fink (deb-based), MacPorts (FreeBSD ports-like), and
 > > NetBSD's pkgsrc are all viable options if you want packaging support
 > > for 3rd-party packages.
 > 
 > Er, excuse me for cutting in but-- that's just not at all the same
 > thing.

[...]

 > I do think its valuable to do so in a way that will integrate with native
 > package managers on those operating systems that they are a native and
 > integrated part of

Why restrict it to those?  That's my point.  If you're going to make
it apt-compatible, you should do all the others, too, because there
are lots of people using them.  Whether or not they are OS-provided.
As far as I'm concerned, asking a Python tool to integrate into any
pms is a non-starter because it really means asking for integration
into all of them.

 > but to say, "Let's not re-create apt!" is a sorrowful stance. It's
 > saying, "Screw Windows, because it isn't as good as what we have." 
 > and "Screw Mac, because its not as good as we have."

It's worse than that.  One point I hoped to make is that "Let's not
recreate apt!" is a way of saying "Screw Windows and Mac and Red Hat
and Gentoo and NetBSD and FreeBSD and ...."

I think it's worth working out a standard format for documenting
dependencies so that a downstream distributor can easily create a
dependency graph (perhaps several of them, since system bootstrap,
package build-time, and package run-time may have different
requirements).  And standard Python distribution media should have a
"manual mode" so that distros can decide where to put things.  But
that's as far as Python itself should ever go; fitting those things
into any given pms is the downstream distro's job.

From pje at telecommunity.com  Thu Apr 10 16:51:13 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 10 Apr 2008 10:51:13 -0400
Subject: [Python-Dev] how to easily consume just the parts of eggs that
 are good for you
In-Reply-To: <7a9c25c20804100012y1a9d797cv1ed4c6d0739c5ba0@mail.gmail.co
 m>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
	<87ej9eic07.fsf@uwakimon.sk.tsukuba.ac.jp>
	<7a9c25c20804100012y1a9d797cv1ed4c6d0739c5ba0@mail.gmail.com>
Message-ID: <20080410145106.051E53A406A@sparrow.telecommunity.com>

At 12:12 AM 4/10/2008 -0700, Stephen Hansen wrote:
>I think PJE's idea here is very good. Just include certain files and 
>such in the RPM/DEB that will satisfy the 
>"python-package-management" system. For RPM/DEB users and their OS's 
>database of packages, its irrelevant largely-- they'll still keep 
>using their own system. But if a product needs something without a 
>.deb or .rpm, or if someone's on an operating system without a 
>native system-- they can still gather everything they need.

I've narrowed it a bit from that, actually.  It's safest if 
easy_install simply refuses to touch any files that it can't tell 
were installed by it (or a compatible system, eg. distutils.)

While that won't solve Paul Moore's desire for the One True Package 
Manager, it will at least make it possible to move forward.


From rbp at isnomore.net  Thu Apr 10 17:10:47 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Thu, 10 Apr 2008 12:10:47 -0300
Subject: [Python-Dev] Next monthly sprint/bugfix day?
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
Message-ID: <20080410151046.GA11041@isnomore.net>

On Wed, Apr 09 2008 at 11:12:58AM BRT, Trent Nelson <tnelson at onresolve.com> wrote:
> Hi,
> 
> Is there another online sprint/bugfix day in the pipeline?  If not, can
> there be? ;-)

+1.

The Sao Paulo User's Group gathered to work on the last Bug Day, and I think
it was very productive (both for generating patches/comments and for us to
get more involved with Python core development).

So, actually, assuming I can speak for at least half of the people from the
PUG who participated in the last Bug Day, +~8 :).


        rbp
-- 
 Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From sergiodj at linux.vnet.ibm.com  Thu Apr 10 19:50:46 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Thu, 10 Apr 2008 14:50:46 -0300
Subject: [Python-Dev] Compiling Python on a biarch (PPC/PPC64)
In-Reply-To: <47FD34D5.2060607@v.loewis.de>
References: <1207773854.5788.23.camel@miki>  <47FD34D5.2060607@v.loewis.de>
Message-ID: <1207849847.6411.6.camel@miki>

Hi Martin,

On Wed, 2008-04-09 at 23:27 +0200, "Martin v. L?wis" wrote:
> > Can anyone give me a hand on this? As far as I could
> > investigate, I must change setup.py in order to get this issue fixed,
> > right?
> 
> Wrong. I didn't read your entire message, but this is definitely wrong.
> setup.py isn't invoked until python itself is built, and
> Modules/python.c is part of python itself.

If I understood correctly, you're saying that Python's building process
involves only the C compiler, right? So, in order to fix this issue that
I've found, I only need to modify configure.in file (and maybe
Makefile.pre.in). Is that correct?

Regards,

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From martin at v.loewis.de  Thu Apr 10 21:23:56 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Apr 2008 21:23:56 +0200
Subject: [Python-Dev] Compiling Python on a biarch (PPC/PPC64)
In-Reply-To: <1207849847.6411.6.camel@miki>
References: <1207773854.5788.23.camel@miki> <47FD34D5.2060607@v.loewis.de>
	<1207849847.6411.6.camel@miki>
Message-ID: <47FE694C.70607@v.loewis.de>

> If I understood correctly, you're saying that Python's building process
> involves only the C compiler, right? So, in order to fix this issue that
> I've found, I only need to modify configure.in file (and maybe
> Makefile.pre.in). Is that correct?

Correct. There are also some additional shell scripts involved, but I
don't think they matter.

Regards,
Martin

From musiccomposition at gmail.com  Fri Apr 11 01:42:41 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 10 Apr 2008 18:42:41 -0500
Subject: [Python-Dev] generated NEWS files
Message-ID: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>

I know that doing merges is rather painful because of the Misc/NEWS
files. Would it be possible to have the NEWS log generated from commit
messages? I was thinking something like this in a message:

Fixed this and that

**NEWS:Library**
#1234 X is now known as Y.

It should be fairly simple to write a script to extract and compile
these messages into NEWS before release.

-- 
Cheers,
Benjamin Peterson

From guido at python.org  Fri Apr 11 02:55:15 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2008 17:55:15 -0700
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
Message-ID: <ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>

In the past we haven't done that because often the change logs are
either too chatty (they do administrative stuff that doesn't deserve a
NEWS entry) or they are too brief (in the heat of the battle
developers don't always write great changelog entries). While it's
easy to fix NEWS its not easy to fix changelog entries (only a few svn
super-users can do it, and it's a pain).

--Guido

On Thu, Apr 10, 2008 at 4:42 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> I know that doing merges is rather painful because of the Misc/NEWS
>  files. Would it be possible to have the NEWS log generated from commit
>  messages? I was thinking something like this in a message:
>
>  Fixed this and that
>
>  **NEWS:Library**
>  #1234 X is now known as Y.
>
>  It should be fairly simple to write a script to extract and compile
>  these messages into NEWS before release.
>
>  --
>  Cheers,
>  Benjamin Peterson
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Fri Apr 11 02:59:35 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 10 Apr 2008 19:59:35 -0500
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
Message-ID: <1afaf6160804101759j2ff0c90ep400cd096ea31f64f@mail.gmail.com>

On Thu, Apr 10, 2008 at 7:55 PM, Guido van Rossum <guido at python.org> wrote:
> In the past we haven't done that because often the change logs are
>  either too chatty (they do administrative stuff that doesn't deserve a
>  NEWS entry) or they are too brief (in the heat of the battle
>  developers don't always write great changelog entries). While it's
>  easy to fix NEWS its not easy to fix changelog entries (only a few svn
>  super-users can do it, and it's a pain).
A NEWs entry wouldn't be added if the special **NEWS** marking wasn't
present. I think we'd have to come up with some way to let people
change/edit by overriding those in the commit messages, but I'm not
sure how that would be done.
>
>  --Guido



-- 
Cheers,
Benjamin Peterson

From guido at python.org  Fri Apr 11 03:24:18 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Apr 2008 18:24:18 -0700
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <1afaf6160804101759j2ff0c90ep400cd096ea31f64f@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
	<1afaf6160804101759j2ff0c90ep400cd096ea31f64f@mail.gmail.com>
Message-ID: <ca471dc20804101824t5254a3cq8361b0c403083a6@mail.gmail.com>

On Thu, Apr 10, 2008 at 5:59 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Thu, Apr 10, 2008 at 7:55 PM, Guido van Rossum <guido at python.org> wrote:
>  > In the past we haven't done that because often the change logs are
>  >  either too chatty (they do administrative stuff that doesn't deserve a
>  >  NEWS entry) or they are too brief (in the heat of the battle
>  >  developers don't always write great changelog entries). While it's
>  >  easy to fix NEWS its not easy to fix changelog entries (only a few svn
>  >  super-users can do it, and it's a pain).

>  A NEWs entry wouldn't be added if the special **NEWS** marking wasn't
>  present. I think we'd have to come up with some way to let people
>  change/edit by overriding those in the commit messages, but I'm not
>  sure how that would be done.

It would be fine if that was *one* way of adding a NEWS entry, but I
don't think it should be the only way, and I don't think it would work
well to regenerate the NEWS file from those entries. In practice we
see corrections on NEWS entries all the time.

Also, there's the issue of which section of the NEWS file an entry
should be added. That's often subjective.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Fri Apr 11 07:53:14 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 10 Apr 2008 22:53:14 -0700
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <ca471dc20804101824t5254a3cq8361b0c403083a6@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
	<1afaf6160804101759j2ff0c90ep400cd096ea31f64f@mail.gmail.com>
	<ca471dc20804101824t5254a3cq8361b0c403083a6@mail.gmail.com>
Message-ID: <52dc1c820804102253i390c9912i6a84bbe074ae8738@mail.gmail.com>

On Thu, Apr 10, 2008 at 6:24 PM, Guido van Rossum <guido at python.org> wrote:

>
> Also, there's the issue of which section of the NEWS file an entry
> should be added. That's often subjective.
>

Agreed.  For example we have both Library and Extension Module categories in
the NEWS file.  I always end up listing changes in the "wrong" one of those
half the time.  In my mind those are the same thing.  A reader is not likely
to care if the change happened to be implemented in the Lib/ directory vs
the Modules/ directory.  From python's point of view they're both just an
import.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080410/62043f0f/attachment.htm 

From martin at v.loewis.de  Fri Apr 11 09:23:34 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Apr 2008 09:23:34 +0200
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
Message-ID: <47FF11F6.5010808@v.loewis.de>

> While it's
> easy to fix NEWS its not easy to fix changelog entries (only a few svn
> super-users can do it, and it's a pain).

Actually, any committer can do that:

svn pe --revprop -rrev svn:log

It may be difficult to remember, but it's fairly easy to execute.

Regards,
Martin

From lists at cheimes.de  Fri Apr 11 11:52:15 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 11 Apr 2008 11:52:15 +0200
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
Message-ID: <ftnccf$ehg$1@ger.gmane.org>

Benjamin Peterson schrieb:
> I know that doing merges is rather painful because of the Misc/NEWS
> files. Would it be possible to have the NEWS log generated from commit
> messages? I was thinking something like this in a message:

No, it's not painful ;) The NEWS files doesn't get merged from trunk to 3.0.

$ svnmerge.py merge
$ svn revert Misc/NEWS

resolve conflicts, compile, test

$ svn ci -F svn....

Christian


From asmodai at in-nomine.org  Fri Apr 11 17:10:24 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Fri, 11 Apr 2008 17:10:24 +0200
Subject: [Python-Dev] Python 2.6a2 execution times with various compilers
Message-ID: <20080411151024.GJ51167@nexus.in-nomine.org>

I did some performance comparisons with various compilers and the resulting
Python 2.6a2 and pybench.

I put the details on
http://www.in-nomine.org/2008/04/11/python-26a2-execution-times-with-various-compilers/

Of course, take benchmark results with a grain of salt, but it seems ICC can
provide people with an added performance edge when using Python. In short: I
measured a speedup between ~21%-29% with ICC compared to GCC.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
I was searching through the Heavens and somehow I slipped...

From status at bugs.python.org  Fri Apr 11 18:06:40 2008
From: status at bugs.python.org (Tracker)
Date: Fri, 11 Apr 2008 18:06:40 +0200 (CEST)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20080411160640.59DC07810A@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (04/04/08 - 04/11/08)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1826 open (+45) / 12616 closed (+21) / 14442 total (+66)

Open issues with patches:   544

Average duration of open issues: 709 days.
Median duration of open issues: 1287 days.

Open Issues Breakdown
   open  1801 (+45)
pending    25 ( +0)

Issues Created Or Reopened (68)
_______________________________

setitimer, getitimer wrapper                                     04/05/08
       http://bugs.python.org/issue2240    reopened loewis                    
       patch                                                                   

Unicode escape sequences not parsed in raw strings.              04/05/08
       http://bugs.python.org/issue2541    reopened gvanrossum                
       patch                                                                   

Python 2.6a2 on Solaris 10 built with SUN C                      04/04/08
CLOSED http://bugs.python.org/issue2551    created  MrJean1                   
                                                                               

test_ctypes failed Python 2.6a2 Solaris 10 SUN C                 04/04/08
       http://bugs.python.org/issue2552    created  MrJean1                   
       patch                                                                   

test_ioctrl failed Python 2.6a2 Solaris 10 SUN C                 04/04/08
CLOSED http://bugs.python.org/issue2553    created  MrJean1                   
                                                                               

test_ioctl failed Python 2.6a2 Solaris 10 SUN C                  04/04/08
       http://bugs.python.org/issue2554    created  MrJean1                   
                                                                               

test_parser failed Python 2.6a2 Solaris 10 SUN C                 04/04/08
CLOSED http://bugs.python.org/issue2555    created  MrJean1                   
                                                                               

changing sys.dont_write_bytecode has not effect                  04/05/08
CLOSED http://bugs.python.org/issue2556    created  benjamin.peterson         
                                                                               

\u and \U in raw strings have regressed in 3.0a4                 04/05/08
CLOSED http://bugs.python.org/issue2557    created  gvanrossum                
                                                                               

Document pickle protocol 3                                       04/05/08
CLOSED http://bugs.python.org/issue2558    created  georg.brandl              
       patch                                                                   

atom sorting error when buiding ctypes                           04/05/08
       http://bugs.python.org/issue2559    created  skip.montanaro            
                                                                               

removal of stale code from myreadline.c                          04/06/08
       http://bugs.python.org/issue2560    created  JosephArmbruster          
       patch                                                                   

Instance remembers old values from former life                   04/06/08
CLOSED http://bugs.python.org/issue2561    created  warnhold                  
                                                                               

Cannot use non-ascii letters in disutils if setuptools is used.  04/06/08
       http://bugs.python.org/issue2562    reopened loewis                    
       patch                                                                   

embed manifest in windows extensions                             04/06/08
       http://bugs.python.org/issue2563    created  mhammond                  
       patch, patch                                                            

Python hangs on FreeBSD7 in test_capi                            04/06/08
CLOSED http://bugs.python.org/issue2564    created  trepan                    
                                                                               

Change 'type' to 'class' in repr/str(builtin-class)              04/07/08
CLOSED http://bugs.python.org/issue2565    created  tjreedy                   
       easy                                                                    

Py3.0a4 wsgiref simple_server failed to start                    04/07/08
       http://bugs.python.org/issue2566    created  delimy                    
                                                                               

Section "New-style and classic classes" needs to be removed/rewr 04/07/08
CLOSED http://bugs.python.org/issue2567    created  loewis                    
                                                                               

Seconds range in time unit                                       04/07/08
       http://bugs.python.org/issue2568    created  datacompboy               
                                                                               

default_scheme in urlparse.urlparse() useless                    04/07/08
       http://bugs.python.org/issue2569    created  pk                        
                                                                               

backport 3.0-style \u/\U processing in raw strings when unicode_ 04/07/08
       http://bugs.python.org/issue2570    created  gvanrossum                
       26backport                                                              

cmd.py always uses raw_input, even when another stdin is specifi 04/07/08
       http://bugs.python.org/issue2571    created  rickbking                 
                                                                               

3.0 pickle docs -- what about old-style classes?                 04/07/08
       http://bugs.python.org/issue2572    created  georg.brandl              
                                                                               

Can't change the framework name on OS X builds                   04/07/08
       http://bugs.python.org/issue2573    created  habnabit                  
       patch                                                                   

Add RFC 3768 SSM Multicast support to "socket"                   04/07/08
       http://bugs.python.org/issue2574    created  bms                       
       patch                                                                   

Reference manual: production for integer literals is missing "bi 04/07/08
CLOSED http://bugs.python.org/issue2575    created  marketdickinson           
       patch, patch                                                            

httplib read() very slow due to lack of socket buffer            04/07/08
       http://bugs.python.org/issue2576    created  reacocard                 
       patch                                                                   

cmd.py should track input file objects so macros with submacros  04/07/08
       http://bugs.python.org/issue2577    created  rickbking                 
                                                                               

Figure out what to do with unittest's redundant APIs             04/08/08
       http://bugs.python.org/issue2578    created  benjamin.peterson         
                                                                               

Misleading 'toctree references unknown document' error           04/08/08
CLOSED http://bugs.python.org/issue2579    created  henryl                    
       patch                                                                   

Revise builtin class int library manual entry                    04/08/08
CLOSED http://bugs.python.org/issue2580    created  tjreedy                   
                                                                               

Vista UAC/elevation support for bdist_wininst                    04/08/08
       http://bugs.python.org/issue2581    created  mhammond                  
       patch, patch                                                            

Unpickling of range objects fail in Py3k                         04/08/08
       http://bugs.python.org/issue2582    created  pythonhacker              
                                                                               

urlparse normalize URL path                                      04/08/08
       http://bugs.python.org/issue2583    created  monk.e.boy                
                                                                               

numeric overflow in IDLE                                         04/08/08
       http://bugs.python.org/issue2584    created  Qodosh                    
                                                                               

urllib2.HTTPError broken due to urllib.addinfourl changes        04/08/08
CLOSED http://bugs.python.org/issue2585    created  djc                       
       patch                                                                   

Integer signedness bugs in zlib modules                          04/08/08
CLOSED http://bugs.python.org/issue2586    created  jnferguson                
                                                                               

PyString_FromStringAndSize() to be considered unsafe             04/10/08
       http://bugs.python.org/issue2587    reopened gvanrossum                
                                                                               

PyOS_vsnprintf() underflow leads to memory corruption            04/08/08
       http://bugs.python.org/issue2588    created  jnferguson                
                                                                               

PyOS_vsnprintf() potential integer overflow leads to memory corr 04/08/08
       http://bugs.python.org/issue2589    created  jnferguson                
                                                                               

S_unpack_from() Read Access Violation                            04/08/08
       http://bugs.python.org/issue2590    created  jnferguson                
                                                                               

ErrorHandler buffer overflow in ?unused? SGI extension module al 04/08/08
       http://bugs.python.org/issue2591    created  jnferguson                
                                                                               

weakref.proxy fails to delegate tp_index slot                    04/08/08
       http://bugs.python.org/issue2592    created  ncoghlan                  
       easy                                                                    

alp_ReadFrames() integer overflow leads to buffer overflow       04/08/08
       http://bugs.python.org/issue2593    created  jnferguson                
                                                                               

alp_readsamps() overflow leads to memory corruption in ?unused?  04/08/08
       http://bugs.python.org/issue2594    created  jnferguson                
                                                                               

Multiple integer overflows in imgfile extension module lead to b 04/08/08
       http://bugs.python.org/issue2595    created  jnferguson                
                                                                               

2to3's fix_xrange needs fix_dict's context information           04/08/08
CLOSED http://bugs.python.org/issue2596    created  collinwinter              
                                                                               

"python2.6 -3" should report list.sort(cmp) as DeprecationWarnin 04/08/08
CLOSED http://bugs.python.org/issue2597    created  dangyogi                  
                                                                               

"{ +(}".format(**{' +(': 44}) should produce ValueError: invalid 04/08/08
       http://bugs.python.org/issue2598    created  dangyogi                  
                                                                               

allow field_name in format strings to default to next positional 04/08/08
       http://bugs.python.org/issue2599    created  dangyogi                  
                                                                               

BindingHTTPConnectionWithTimeout and BindingHTTPHandlerWithTimeo 04/08/08
       http://bugs.python.org/issue2600    created  pau                       
                                                                               

[regression] reading from a urllib2 file descriptor happens byte 04/08/08
       http://bugs.python.org/issue2601    created  doko                      
                                                                               

importing gtk hangs (freebsd 7.0)                                04/09/08
CLOSED http://bugs.python.org/issue2602    created  trepan                    
                                                                               

Make range __eq__ work                                           04/09/08
       http://bugs.python.org/issue2603    created  benjamin.peterson         
       patch                                                                   

doctest.DocTestCase fails when run repeatedly                    04/10/08
       http://bugs.python.org/issue2604    created  pjd                       
                                                                               

Descriptor instance attributes not interpreted consistently      04/10/08
       http://bugs.python.org/issue2605    created  pjd                       
                                                                               

trace module crashes due to using wrong sort idiom               04/10/08
CLOSED http://bugs.python.org/issue2606    created  mark                      
       patch                                                                   

why is (default)dict so slow on tuples???                        04/10/08
CLOSED http://bugs.python.org/issue2607    created  eisele                    
                                                                               

manpages for sphinx-build & sphinx-quickstart                    04/10/08
       http://bugs.python.org/issue2608    created  dottedmag                 
                                                                               

Tests fail if ./@test is not writeable                           04/10/08
       http://bugs.python.org/issue2609    created  belopolsky                
                                                                               

string representation of range                                   04/10/08
       http://bugs.python.org/issue2610    created  bmiller                   
       patch                                                                   

Extend buildbot web interface to allow for forced tests to be ru 04/10/08
       http://bugs.python.org/issue2611    created  Trent.Nelson              
                                                                               

file.tell() returns Long usually, Int if subclassed              04/11/08
CLOSED http://bugs.python.org/issue2612    created  esrever_otua              
                                                                               

inconsistency with bare * in parameter list                      04/11/08
       http://bugs.python.org/issue2613    created  mark                      
                                                                               

Console UnicodeDecodeError s once more                           04/11/08
       http://bugs.python.org/issue2614    created  techtonik                 
                                                                               

xml.dom.minidom documentation consistency and update             04/11/08
       http://bugs.python.org/issue2615    created  asmodai                   
       patch                                                                   

ctypes.pointer(), ctypes.POINTER() speedup                       04/11/08
       http://bugs.python.org/issue2616    created  theller                   
       patch, patch                                                            



Issues Now Closed (47)
______________________

socket.socket.getsockname() has inconsistent UNIX/Windows behavi  226 days
       http://bugs.python.org/issue1049    Trent.Nelson              
                                                                               

pyconfig.h not compatible with MS VC++ Express Edition            175 days
       http://bugs.python.org/issue1297    fdrake                    
                                                                               

sys.argv is wrong for unicode strings                              51 days
       http://bugs.python.org/issue2128    benjamin.peterson         
       patch                                                                   

tokenize: does not allow CR for a newline                          43 days
       http://bugs.python.org/issue2182    jaredgrubb                
                                                                               

Further simplify dict literal bytecode                             41 days
       http://bugs.python.org/issue2197    rhettinger                
       patch                                                                   

Py30a3: eval in threaded code raises SystemError                   38 days
       http://bugs.python.org/issue2221    amaury.forgeotdarc        
                                                                               

Install failure of 2.6a1 on Windows XP without VS8 installed       34 days
       http://bugs.python.org/issue2256    loewis                    
                                                                               

[PATCH] Tcl/Tk 8.4.16 patches needed to get an x64 Windows build   20 days
       http://bugs.python.org/issue2296    Trent.Nelson              
       patch, 64bit                                                            

Compiler warnings when using UCS4                                  20 days
       http://bugs.python.org/issue2388    loewis                    
       easy                                                                    

types module can be implemented only in Python                     21 days
       http://bugs.python.org/issue2408    amaury.forgeotdarc        
       patch                                                                   

absolute import doesn't work for standard python modules           21 days
       http://bugs.python.org/issue2410    ncoghlan                  
                                                                               

Issues with getargs_n() and PyNumber_Index.                        21 days
       http://bugs.python.org/issue2440    benjamin.peterson         
       patch, patch, 64bit                                                     

Line tracing of continue after always-taken if is incorrect         7 days
       http://bugs.python.org/issue2506    tjreedy                   
                                                                               

64bit cross compilation on windows                                 10 days
       http://bugs.python.org/issue2513    loewis                    
       patch, 64bit                                                            

Segfault while operating on closed sqlite3 cursor.                  7 days
       http://bugs.python.org/issue2515    pjdavis                   
                                                                               

class USTimeZone in Doc/includes/tzinfo-examples.py is out of da    7 days
       http://bugs.python.org/issue2525    georg.brandl              
       patch, easy                                                             

Document IO module                                                  8 days
       http://bugs.python.org/issue2530    benjamin.peterson         
       patch                                                                   

sphinx.ext.autodoc fails to expand tabs in docstrings               2 days
       http://bugs.python.org/issue2545    georg.brandl              
                                                                               

Python-2.5.2: crash in visit_decref () at Modules/gcmodule.c:270    1 days
       http://bugs.python.org/issue2546    garikvm                   
                                                                               

Python 2.6a2 on Solaris 10 built with SUN C                         0 days
       http://bugs.python.org/issue2551    loewis                    
                                                                               

test_ioctrl failed Python 2.6a2 Solaris 10 SUN C                    0 days
       http://bugs.python.org/issue2553    loewis                    
                                                                               

test_parser failed Python 2.6a2 Solaris 10 SUN C                    1 days
       http://bugs.python.org/issue2555    benjamin.peterson         
                                                                               

changing sys.dont_write_bytecode has not effect                     0 days
       http://bugs.python.org/issue2556    benjamin.peterson         
                                                                               

\u and \U in raw strings have regressed in 3.0a4                    2 days
       http://bugs.python.org/issue2557    gvanrossum                
                                                                               

Document pickle protocol 3                                          1 days
       http://bugs.python.org/issue2558    georg.brandl              
       patch                                                                   

Instance remembers old values from former life                      0 days
       http://bugs.python.org/issue2561    georg.brandl              
                                                                               

Python hangs on FreeBSD7 in test_capi                               2 days
       http://bugs.python.org/issue2564    amaury.forgeotdarc        
                                                                               

Change 'type' to 'class' in repr/str(builtin-class)                 0 days
       http://bugs.python.org/issue2565    loewis                    
       easy                                                                    

Section "New-style and classic classes" needs to be removed/rewr    1 days
       http://bugs.python.org/issue2567    georg.brandl              
                                                                               

Reference manual: production for integer literals is missing "bi    2 days
       http://bugs.python.org/issue2575    georg.brandl              
       patch, patch                                                            

Misleading 'toctree references unknown document' error              2 days
       http://bugs.python.org/issue2579    georg.brandl              
       patch                                                                   

Revise builtin class int library manual entry                       2 days
       http://bugs.python.org/issue2580    georg.brandl              
                                                                               

urllib2.HTTPError broken due to urllib.addinfourl changes           1 days
       http://bugs.python.org/issue2585    georg.brandl              
       patch                                                                   

Integer signedness bugs in zlib modules                             2 days
       http://bugs.python.org/issue2586    belopolsky                
                                                                               

2to3's fix_xrange needs fix_dict's context information              0 days
       http://bugs.python.org/issue2596    collinwinter              
                                                                               

"python2.6 -3" should report list.sort(cmp) as DeprecationWarnin    0 days
       http://bugs.python.org/issue2597    rhettinger                
                                                                               

importing gtk hangs (freebsd 7.0)                                   0 days
       http://bugs.python.org/issue2602    benjamin.peterson         
                                                                               

trace module crashes due to using wrong sort idiom                  0 days
       http://bugs.python.org/issue2606    loewis                    
       patch                                                                   

why is (default)dict so slow on tuples???                           1 days
       http://bugs.python.org/issue2607    eisele                    
                                                                               

file.tell() returns Long usually, Int if subclassed                 0 days
       http://bugs.python.org/issue2612    benjamin.peterson         
                                                                               

urllib.urlopen results.readline is slow                          2265 days
       http://bugs.python.org/issue508157  reacocard                 
                                                                               

file (& socket) I/O are not thread safe                          2061 days
       http://bugs.python.org/issue595601  gregory.p.smith           
                                                                               

thread unsafe file objects cause crash                           1650 days
       http://bugs.python.org/issue815646  gregory.p.smith           
       patch                                                                   

Tkdnd.py crashes due to read-only attributes                     1116 days
       http://bugs.python.org/issue1164742 benjamin.peterson         
                                                                               

non-admin install may fail (win xp pro)                          1007 days
       http://bugs.python.org/issue1232947 loewis                    
                                                                               

Missing builtin help for with and as                              610 days
       http://bugs.python.org/issue1536059 ncoghlan                  
                                                                               

Add RegEnableReflectionKey and RegDisableReflectionKey            268 days
       http://bugs.python.org/issue1753245 mhammond                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 24 PyString_FromStringAndSize() to be considered unsafe               1 days
open    http://bugs.python.org/issue2587   

 18 Cannot use non-ascii letters in disutils if setuptools is used.    5 days
open    http://bugs.python.org/issue2562   

 17 Unicode escape sequences not parsed in raw strings.                6 days
open    http://bugs.python.org/issue2541   

 15 test_ctypes failed Python 2.6a2 Solaris 10 SUN C                   7 days
open    http://bugs.python.org/issue2552   

 14 Missing *-unpacking generalizations                               27 days
open    http://bugs.python.org/issue2292   

 10 SO_REUSEADDR doesn't have the same semantics on Windows as on U    7 days
open    http://bugs.python.org/issue2550   

  9 string representation of range                                     1 days
open    http://bugs.python.org/issue2610   

  9 Make range __eq__ work                                             2 days
open    http://bugs.python.org/issue2603   

  9 Document IO module                                                 8 days
closed  http://bugs.python.org/issue2530   

  8 PyOS_vsnprintf() underflow leads to memory corruption              3 days
open    http://bugs.python.org/issue2588   




From divinekid at gmail.com  Fri Apr 11 18:35:47 2008
From: divinekid at gmail.com (Haoyu Bai)
Date: Sat, 12 Apr 2008 00:35:47 +0800
Subject: [Python-Dev] Need help for SWIG's Python 3.0 backend
Message-ID: <47FF9363.7060005@gmail.com>

Hello,

I am a Google Summer of Code student who preparing a SWIG's Python 3.0 
support proposal. Here's detail of my proposal:

http://www.dabeaz.com/cgi-bin/wiki.pl?GSoCPython3Proposal

And abstract shown below for convenient:

This project adds Python 3.0 support for SWIG. We will add a "-3" option 
to SWIG's current backend, which indicates SWIG to generate wrapper for 
Python 3. We also make SWIG generate more efficient code and more clear 
proxy by utilizing Python 3's new features.

The considered features are as follows:

     * Function Annotations

     * Mutable Buffer Support

     * Abstract Base Classes


I have read PEPs and Python 3's document, then did some experiment on 
the API. I have modified a SWIG generated wrapper code by hand so it can 
  running with Python 3.0.

However, there still some API changes I can't handle. SWIG used some 
undocumented C API, for example the _PyInstance_Lookup(). And some API 
disappeared, I can't found the alternative of them, for example 
PyInstance_NewRaw().

I think I will need a lot of help from Python developers if my proposal 
is accepted. So I post this here to make sure if I can get help when 
doing this project. And I really appreciate if you can give any advice 
about how to solve the problems I mentioned before.

Thank you!


Best regards,

Haoyu Bai
4/12/2008

From sergiodj at linux.vnet.ibm.com  Fri Apr 11 19:10:02 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Fri, 11 Apr 2008 14:10:02 -0300
Subject: [Python-Dev] Python 32- and 64-bit living together
Message-ID: <1207933802.6420.6.camel@miki>

Hi all,

My question is simple: is there any problem when installing/using both
32- and 64-bit Python's on the same machine? I'm more concerned about
header files (those installed under /usr/include/python-2.x), because as
far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
something on them.

Thanks,

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From fwierzbicki at gmail.com  Fri Apr 11 19:38:57 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 11 Apr 2008 13:38:57 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
Message-ID: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>

Hi all,

I was wondering if it might be possible for Jython to get an "arg
space" for command line execution.  We try to deliver the same
switches that Python delivers (so for example -c means "program passed
in as string" on Python and Jython).  We have some need for arguments
that would be Jython-specific, for example we would like to be able to
pass arguments to the Java process that starts Jython.  I would just
make one up, but it would be best if it was future compatible with
Python (so -J would be great, but it would be annoying if -J started
to mean something in a future Python).

So what do guys say?  Can we hava -J?  I don't think it is being used yet...

-Frank

From guido at python.org  Fri Apr 11 19:40:50 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Apr 2008 10:40:50 -0700
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
Message-ID: <ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>

Works for me. We should have a patch to CPython that looks for -J and
rejects it with "-J is reserved for Jython".

On Fri, Apr 11, 2008 at 10:38 AM, Frank Wierzbicki
<fwierzbicki at gmail.com> wrote:
> Hi all,
>
>  I was wondering if it might be possible for Jython to get an "arg
>  space" for command line execution.  We try to deliver the same
>  switches that Python delivers (so for example -c means "program passed
>  in as string" on Python and Jython).  We have some need for arguments
>  that would be Jython-specific, for example we would like to be able to
>  pass arguments to the Java process that starts Jython.  I would just
>  make one up, but it would be best if it was future compatible with
>  Python (so -J would be great, but it would be annoying if -J started
>  to mean something in a future Python).
>
>  So what do guys say?  Can we hava -J?  I don't think it is being used yet...
>
>  -Frank
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fwierzbicki at gmail.com  Fri Apr 11 19:43:39 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 11 Apr 2008 13:43:39 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
Message-ID: <4dab5f760804111043x998c271yce8b5da8a909b8d4@mail.gmail.com>

On Fri, Apr 11, 2008 at 1:40 PM, Guido van Rossum <guido at python.org> wrote:
> Works for me. We should have a patch to CPython that looks for -J and
>  rejects it with "-J is reserved for Jython".
Great!  Knowing this crowd it is probably already implemented -- but
if not I'd love to dust off my C skills and upload a patch, I doubt I
would cause harm in this area of Python :)

-Frank

From mal at egenix.com  Fri Apr 11 19:38:18 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Apr 2008 19:38:18 +0200
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <1207933802.6420.6.camel@miki>
References: <1207933802.6420.6.camel@miki>
Message-ID: <47FFA20A.4070606@egenix.com>

On 2008-04-11 19:10, S?rgio Durigan J?nior wrote:
> Hi all,
> 
> My question is simple: is there any problem when installing/using both
> 32- and 64-bit Python's on the same machine? I'm more concerned about
> header files (those installed under /usr/include/python-2.x), because as
> far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
> something on them.

The include files are all static and can be used on both 32-bit and
64-bit platforms or installations.

Only the /usr/lib/python2.x files differ between 32-bit and 64-bit
(the configuration files are in /usr/lib/python2.x/config).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 11 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From sergiodj at linux.vnet.ibm.com  Fri Apr 11 20:21:44 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Fri, 11 Apr 2008 15:21:44 -0300
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <47FFA20A.4070606@egenix.com>
References: <1207933802.6420.6.camel@miki>  <47FFA20A.4070606@egenix.com>
Message-ID: <1207938104.6420.15.camel@miki>

Hi Lemburg,

On Fri, 2008-04-11 at 19:38 +0200, M.-A. Lemburg wrote:
> On 2008-04-11 19:10, S?rgio Durigan J?nior wrote:
> > Hi all,
> > 
> > My question is simple: is there any problem when installing/using both
> > 32- and 64-bit Python's on the same machine? I'm more concerned about
> > header files (those installed under /usr/include/python-2.x), because as
> > far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
> > something on them.
> 
> The include files are all static and can be used on both 32-bit and
> 64-bit platforms or installations.

Thanks :-).

> Only the /usr/lib/python2.x files differ between 32-bit and 64-bit
> (the configuration files are in /usr/lib/python2.x/config).

Hmm, right. I tried to modify the installation path (using --libdir
in ./configure) to /usr/lib64, but some *.pyo objects still are
installed under /usr/lib. AFAIK, these objects are bitness-dependent
(i.e., if they were generated by a 32-bit Python, they can only be
execute by a 32-bit Python - and vice-versa), right?

Is there any way to separate these arch-dependent files in /usr/lib
and /usr/lib64 depending on their bitness?

Thanks,

P.S.: I think this "misbehaviour" of --libdir is a bug. IMHO, it should
put every arch-dependent file in the path that the user provided.

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From fwierzbicki at gmail.com  Fri Apr 11 20:49:17 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 11 Apr 2008 14:49:17 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804111043x998c271yce8b5da8a909b8d4@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<4dab5f760804111043x998c271yce8b5da8a909b8d4@mail.gmail.com>
Message-ID: <4dab5f760804111149y5357e875nbbadf3bdd593673@mail.gmail.com>

Patch is here: http://bugs.python.org/issue2617

-Frank

From brett at python.org  Fri Apr 11 20:51:18 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Apr 2008 20:51:18 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
Message-ID: <bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>

On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <guido at python.org> wrote:
> Works for me. We should have a patch to CPython that looks for -J and
>  rejects it with "-J is reserved for Jython".
>

Do we want it to be Jython-specific, or should it be available to any
alternative VM? I don't know if the IronPython folks need anything for
.NET, but maybe they would like one?

-Brett

>
>
>  On Fri, Apr 11, 2008 at 10:38 AM, Frank Wierzbicki
>  <fwierzbicki at gmail.com> wrote:
>  > Hi all,
>  >
>  >  I was wondering if it might be possible for Jython to get an "arg
>  >  space" for command line execution.  We try to deliver the same
>  >  switches that Python delivers (so for example -c means "program passed
>  >  in as string" on Python and Jython).  We have some need for arguments
>  >  that would be Jython-specific, for example we would like to be able to
>  >  pass arguments to the Java process that starts Jython.  I would just
>  >  make one up, but it would be best if it was future compatible with
>  >  Python (so -J would be great, but it would be annoying if -J started
>  >  to mean something in a future Python).
>  >
>  >  So what do guys say?  Can we hava -J?  I don't think it is being used yet...
>  >
>  >  -Frank
>  >  _______________________________________________
>  >  Python-Dev mailing list
>  >  Python-Dev at python.org
>  >  http://mail.python.org/mailman/listinfo/python-dev
>  >  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>  >
>
>
>
>  --
>  --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From fwierzbicki at gmail.com  Fri Apr 11 21:13:44 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 11 Apr 2008 15:13:44 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
Message-ID: <4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>

On Fri, Apr 11, 2008 at 2:51 PM, Brett Cannon <brett at python.org> wrote:
> On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <guido at python.org> wrote:
>
> > Works for me. We should have a patch to CPython that looks for -J and
>  >  rejects it with "-J is reserved for Jython".
>  >
>
>  Do we want it to be Jython-specific, or should it be available to any
>  alternative VM? I don't know if the IronPython folks need anything for
>  .NET, but maybe they would like one?
-X was suggested on Jython's irc.  I kind of like -J, but -X would
work for us too.

-Frank

From dinov at exchange.microsoft.com  Fri Apr 11 21:39:58 2008
From: dinov at exchange.microsoft.com (Dino Viehland)
Date: Fri, 11 Apr 2008 12:39:58 -0700
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
Message-ID: <7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>

IronPython already uses -X:OptionName for special IronPython only options so +1 for -X.

-----Original Message-----
From: python-dev-bounces+dinov=microsoft.com at python.org [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of Frank Wierzbicki
Sent: Friday, April 11, 2008 12:14 PM
To: Brett Cannon
Cc: Guido van Rossum; python-dev at python.org
Subject: Re: [Python-Dev] Reserving an arg space for Jython

On Fri, Apr 11, 2008 at 2:51 PM, Brett Cannon <brett at python.org> wrote:
> On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <guido at python.org> wrote:
>
> > Works for me. We should have a patch to CPython that looks for -J and
>  >  rejects it with "-J is reserved for Jython".
>  >
>
>  Do we want it to be Jython-specific, or should it be available to any
>  alternative VM? I don't know if the IronPython folks need anything for
>  .NET, but maybe they would like one?
-X was suggested on Jython's irc.  I kind of like -J, but -X would
work for us too.

-Frank
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com

From fuzzyman at voidspace.org.uk  Fri Apr 11 21:52:36 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 11 Apr 2008 20:52:36 +0100
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
Message-ID: <47FFC184.5000009@voidspace.org.uk>

Frank Wierzbicki wrote:
> On Fri, Apr 11, 2008 at 2:51 PM, Brett Cannon <brett at python.org> wrote:
>   
>> On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <guido at python.org> wrote:
>>
>>     
>>> Works for me. We should have a patch to CPython that looks for -J and
>>>       
>>  >  rejects it with "-J is reserved for Jython".
>>  >
>>
>>  Do we want it to be Jython-specific, or should it be available to any
>>  alternative VM? I don't know if the IronPython folks need anything for
>>  .NET, but maybe they would like one?
>>     
> -X was suggested on Jython's irc.  I kind of like -J, but -X would
> work for us too.
>   

IronPython has a host of -X:Something command line switches - so 
reserving -X would be helpful.

Michael Foord

> -Frank
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


From nnorwitz at gmail.com  Fri Apr 11 22:03:25 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 11 Apr 2008 13:03:25 -0700
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
Message-ID: <ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>

I was also going to suggest a platform independent option.  I like
-Xwhat-follows-is-impl-dependent.

n

On Fri, Apr 11, 2008 at 12:39 PM, Dino Viehland
<dinov at exchange.microsoft.com> wrote:
> IronPython already uses -X:OptionName for special IronPython only options so +1 for -X.
>
>
>
>  -----Original Message-----
>  From: python-dev-bounces+dinov=microsoft.com at python.org [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of Frank Wierzbicki
>  Sent: Friday, April 11, 2008 12:14 PM
>  To: Brett Cannon
>  Cc: Guido van Rossum; python-dev at python.org
>  Subject: Re: [Python-Dev] Reserving an arg space for Jython
>
>  On Fri, Apr 11, 2008 at 2:51 PM, Brett Cannon <brett at python.org> wrote:
>  > On Fri, Apr 11, 2008 at 7:40 PM, Guido van Rossum <guido at python.org> wrote:
>  >
>  > > Works for me. We should have a patch to CPython that looks for -J and
>  >  >  rejects it with "-J is reserved for Jython".
>  >  >
>  >
>  >  Do we want it to be Jython-specific, or should it be available to any
>  >  alternative VM? I don't know if the IronPython folks need anything for
>  >  .NET, but maybe they would like one?
>  -X was suggested on Jython's irc.  I kind of like -J, but -X would
>  work for us too.
>
>  -Frank
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/dinov%40microsoft.com
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From mal at egenix.com  Fri Apr 11 22:06:19 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Apr 2008 22:06:19 +0200
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <1207938104.6420.15.camel@miki>
References: <1207933802.6420.6.camel@miki> <47FFA20A.4070606@egenix.com>
	<1207938104.6420.15.camel@miki>
Message-ID: <47FFC4BB.3070209@egenix.com>

On 2008-04-11 20:21, S?rgio Durigan J?nior wrote:
> Hi Lemburg,
> 
> On Fri, 2008-04-11 at 19:38 +0200, M.-A. Lemburg wrote:
>> On 2008-04-11 19:10, S?rgio Durigan J?nior wrote:
>>> Hi all,
>>>
>>> My question is simple: is there any problem when installing/using both
>>> 32- and 64-bit Python's on the same machine? I'm more concerned about
>>> header files (those installed under /usr/include/python-2.x), because as
>>> far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
>>> something on them.
>> The include files are all static and can be used on both 32-bit and
>> 64-bit platforms or installations.
> 
> Thanks :-).
> 
>> Only the /usr/lib/python2.x files differ between 32-bit and 64-bit
>> (the configuration files are in /usr/lib/python2.x/config).
> 
> Hmm, right. I tried to modify the installation path (using --libdir
> in ./configure) to /usr/lib64, but some *.pyo objects still are
> installed under /usr/lib. AFAIK, these objects are bitness-dependent
> (i.e., if they were generated by a 32-bit Python, they can only be
> execute by a 32-bit Python - and vice-versa), right?

Right.

> Is there any way to separate these arch-dependent files in /usr/lib
> and /usr/lib64 depending on their bitness?

There's no need for that. Only the config/ dir which is included
in the Python lib dir is dependent on the Python configuration.

> Thanks,
> 
> P.S.: I think this "misbehaviour" of --libdir is a bug. IMHO, it should
> put every arch-dependent file in the path that the user provided.

You should probably have a look at how RedHat or openSUSE solve
these problems. Some of them have patched Python to fit their
needs. You may have to do that as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 11 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From sergiodj at linux.vnet.ibm.com  Fri Apr 11 22:25:08 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Fri, 11 Apr 2008 17:25:08 -0300
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <47FFC4BB.3070209@egenix.com>
References: <1207933802.6420.6.camel@miki>  <47FFA20A.4070606@egenix.com>
	<1207938104.6420.15.camel@miki>  <47FFC4BB.3070209@egenix.com>
Message-ID: <1207945509.6420.30.camel@miki>

On Fri, 2008-04-11 at 22:06 +0200, M.-A. Lemburg wrote:
 
> > Hmm, right. I tried to modify the installation path (using --libdir
> > in ./configure) to /usr/lib64, but some *.pyo objects still are
> > installed under /usr/lib. AFAIK, these objects are bitness-dependent
> > (i.e., if they were generated by a 32-bit Python, they can only be
> > execute by a 32-bit Python - and vice-versa), right?
> 
> Right.
> 
> > Is there any way to separate these arch-dependent files in /usr/lib
> > and /usr/lib64 depending on their bitness?
> 
> There's no need for that. Only the config/ dir which is included
> in the Python lib dir is dependent on the Python configuration.


I'm afraid I still don't understand your point. I mean, if the *.pyo
file *is* dependent on the bitness of the Python interpreter (as you
confirmed in my first question), therefore when I decide to have both
32- and 64-bit Python on my system I *must* have two versions of
every .pyo file: one for 32- and another for 64-bit Python. What I've
missed?


> You should probably have a look at how RedHat or openSUSE solve
> these problems. Some of them have patched Python to fit their
> needs. You may have to do that as well.

I'll sure take a look at them. Thanks!

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From mal at egenix.com  Fri Apr 11 22:42:09 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Apr 2008 22:42:09 +0200
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <1207945509.6420.30.camel@miki>
References: <1207933802.6420.6.camel@miki>
	<47FFA20A.4070606@egenix.com>	<1207938104.6420.15.camel@miki>
	<47FFC4BB.3070209@egenix.com> <1207945509.6420.30.camel@miki>
Message-ID: <47FFCD21.1060206@egenix.com>

On 2008-04-11 22:25, S?rgio Durigan J?nior wrote:
> On Fri, 2008-04-11 at 22:06 +0200, M.-A. Lemburg wrote:
>  
>>> Hmm, right. I tried to modify the installation path (using --libdir
>>> in ./configure) to /usr/lib64, but some *.pyo objects still are
>>> installed under /usr/lib. AFAIK, these objects are bitness-dependent
>>> (i.e., if they were generated by a 32-bit Python, they can only be
>>> execute by a 32-bit Python - and vice-versa), right?
>> Right.

Sorry, I misread you question. PYO and PYC files are *not* dependent
on 32/64 bit sizes.

>>> Is there any way to separate these arch-dependent files in /usr/lib
>>> and /usr/lib64 depending on their bitness?
>> There's no need for that. Only the config/ dir which is included
>> in the Python lib dir is dependent on the Python configuration.
> 
> 
> I'm afraid I still don't understand your point. I mean, if the *.pyo
> file *is* dependent on the bitness of the Python interpreter (as you
> confirmed in my first question), therefore when I decide to have both
> 32- and 64-bit Python on my system I *must* have two versions of
> every .pyo file: one for 32- and another for 64-bit Python. What I've
> missed?

Sorry for the confusion.

>> You should probably have a look at how RedHat or openSUSE solve
>> these problems. Some of them have patched Python to fit their
>> needs. You may have to do that as well.
> 
> I'll sure take a look at them. Thanks!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 11 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From musiccomposition at gmail.com  Fri Apr 11 23:28:56 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 11 Apr 2008 16:28:56 -0500
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <47FF11F6.5010808@v.loewis.de>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
	<47FF11F6.5010808@v.loewis.de>
Message-ID: <1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>

On Fri, Apr 11, 2008 at 2:23 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > While it's
>  > easy to fix NEWS its not easy to fix changelog entries (only a few svn
>  > super-users can do it, and it's a pain).
>
>  Actually, any committer can do that:
>
>  svn pe --revprop -rrev svn:log
Wow, I'm adding that to my list of things I never dreamed Subversion
could do! :P

I just tried it, and although it works, I get this output:
svn: 'post-revprop-change' hook failed; no error output available

Significant?
>
>  It may be difficult to remember, but it's fairly easy to execute.
>
>  Regards,
>  Martin
>



-- 
Cheers,
Benjamin Peterson

From martin at v.loewis.de  Fri Apr 11 23:53:20 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 11 Apr 2008 23:53:20 +0200
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>	
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>	
	<47FF11F6.5010808@v.loewis.de>
	<1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>
Message-ID: <47FFDDD0.10900@v.loewis.de>

> I just tried it, and although it works, I get this output:
> svn: 'post-revprop-change' hook failed; no error output available
> 
> Significant?

It's the mailer.py hook failing. I'm not quite sure why it fails,
and had no time to look it up. It reads

#!/bin/sh
REPOS="$1"
REV="$2"
USER="$3"
PROPNAME="$4"

/data/repos/projects/hooks/mailer.py propchange "$@"

The script is the same mailer.py that is also invoked in post-commit.
Can anybody see a problem with that?

Regards,
Martin

From brett at python.org  Fri Apr 11 23:54:36 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Apr 2008 14:54:36 -0700
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
	<47FF11F6.5010808@v.loewis.de>
	<1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>
Message-ID: <bbaeab100804111454k59d85f05he636f60efccae047@mail.gmail.com>

On Fri, Apr 11, 2008 at 2:28 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Apr 11, 2008 at 2:23 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>  > > While it's
>  >  > easy to fix NEWS its not easy to fix changelog entries (only a few svn
>  >  > super-users can do it, and it's a pain).
>  >
>  >  Actually, any committer can do that:
>  >
>  >  svn pe --revprop -rrev svn:log
>  Wow, I'm adding that to my list of things I never dreamed Subversion
>  could do! :P

Then you should read http://python.org/dev/faq/ which is essentially a
svn FAQ for Python development.

-Brett

From martin at v.loewis.de  Fri Apr 11 23:58:37 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Apr 2008 23:58:37 +0200
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <1207933802.6420.6.camel@miki>
References: <1207933802.6420.6.camel@miki>
Message-ID: <47FFDF0D.6090307@v.loewis.de>

> My question is simple:

It's also off-topic for python-dev, which is about the development
of Python, not the development with Python.

> is there any problem when installing/using both
> 32- and 64-bit Python's on the same machine? I'm more concerned about
> header files (those installed under /usr/include/python-2.x), because as
> far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
> something on them.

You can't install them both into the same prefix (obviously, as they
would overwrite their binary files).

With two installation, use the header files you got from the 32-bit
install for 32-bit extensions/embedding, and the header files you
got from the 64-bit install for 64-bit extensions/embedding. The
only header file that will actually differ is pyconfig.h.

Regards,
Martin

From sergiodj at linux.vnet.ibm.com  Sat Apr 12 00:24:17 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Fri, 11 Apr 2008 19:24:17 -0300
Subject: [Python-Dev] Python 32- and 64-bit living together
In-Reply-To: <47FFDF0D.6090307@v.loewis.de>
References: <1207933802.6420.6.camel@miki>  <47FFDF0D.6090307@v.loewis.de>
Message-ID: <1207952657.6420.43.camel@miki>

Hi Martin,

On Fri, 2008-04-11 at 23:58 +0200, "Martin v. L?wis" wrote:
> > My question is simple:
> 
> It's also off-topic for python-dev, which is about the development
> of Python, not the development with Python.

With all respect, I think you're confusing things. I'm here trying to
find some solutions for Python's build system (which, AFAIK, is about
the development *of* Python), which I think is bogus on some systems.
I'm just trying to better understand how it works here, and maybe send
some patches if I find some solution for my problems.

> > is there any problem when installing/using both
> > 32- and 64-bit Python's on the same machine? I'm more concerned about
> > header files (those installed under /usr/include/python-2.x), because as
> > far as I could see there's nothing similar to a "#ifdef USE_64BIT" or
> > something on them.
> 
> You can't install them both into the same prefix (obviously, as they
> would overwrite their binary files).

I was thinking about renaming the binary files, since they're
(apparently) the only things that get overwritten *and* are dependent of
the bit size (32/64). The rest of the files that would be overwritten,
AFAIK, is not dependent of bitness so there's no problem at all. I think
this may work (actually, I did the same thing successfuly with Perl).

> 
> With two installation, use the header files you got from the 32-bit
> install for 32-bit extensions/embedding, and the header files you
> got from the 64-bit install for 64-bit extensions/embedding. The
> only header file that will actually differ is pyconfig.h.

If I correctly understand what you're saying, you're proposing
installing the header files in different places, right? Since these
files are the same (except for pyconfig.h, as you said), I assume that
the reason for this separation is only for safety. I'll consider your
proposal.

Regards,

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From greg at krypto.org  Sat Apr 12 03:31:10 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 11 Apr 2008 18:31:10 -0700
Subject: [Python-Dev] strange import encodings error in trunk?
Message-ID: <52dc1c820804111831n78b55960kddf51011f01953bf@mail.gmail.com>

has anyone ever seen this error?  this is a pristine --with-pydebug build of
trunk:

>>> msg = 'ABC'
>>> x = msg.decode('utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/gps/python/trunk/Lib/encodings/__init__.py", line 100, in
search_function
    level=0)
TypeError: SetupPathsAndImport() got an unexpected keyword argument 'level'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080411/0283d266/attachment.htm 

From thomas at python.org  Sat Apr 12 04:01:48 2008
From: thomas at python.org (Thomas Wouters)
Date: Sat, 12 Apr 2008 04:01:48 +0200
Subject: [Python-Dev] strange import encodings error in trunk?
In-Reply-To: <52dc1c820804111831n78b55960kddf51011f01953bf@mail.gmail.com>
References: <52dc1c820804111831n78b55960kddf51011f01953bf@mail.gmail.com>
Message-ID: <9e804ac0804111901j1cde50abj6761636e5276ade3@mail.gmail.com>

On Sat, Apr 12, 2008 at 3:31 AM, Gregory P. Smith <greg at krypto.org> wrote:

> has anyone ever seen this error?  this is a pristine --with-pydebug build
> of trunk:
>
> >>> msg = 'ABC'
> >>> x = msg.decode('utf8')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/gps/python/trunk/Lib/encodings/__init__.py", line 100, in
> search_function
>     level=0)
> TypeError: SetupPathsAndImport() got an unexpected keyword argument
> 'level'
>

Your Google sitecustomize is in the way :) Run python with -E.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080412/5dfe1a5e/attachment.htm 

From zooko at zooko.com  Sat Apr 12 06:02:04 2008
From: zooko at zooko.com (zooko)
Date: Fri, 11 Apr 2008 21:02:04 -0700
Subject: [Python-Dev] an example of setuptools being used to good effect --
	allmydata.org Tahoe
References: <E921074D-3D11-43DD-A2C1-ECB86B0C842D@zooko.com>
Message-ID: <3C9044AE-D292-41B6-976A-4313CA8AC174@zooko.com>

Folks:

I'm sorry, but I am not caught up on the current conversation about  
packaging.  I'm very busy with exciting Python development -- http:// 
allmydata.com and http://allmydata.org .  I understand from PJE's  
message that he thinks I misunderstand some things about PEP 262;  
this is entirely possible.  I intend to catch up on reading the  
emails of this conversation and to read carefully PJE's messages  
about PEP 262 in the coming days.

Meanwhile, here is the last message that I wrote before I got swamped  
with the aforementioned excitement:


On Apr 9, 2008, at 5:59 PM, Greg Ewing wrote:
> Paul Moore wrote:
>
>> I believe that Mac OS X goes for an even simpler structure -
>> applications store *everything* in the one directory, so that
>> install/uninstall is simply a directory copy/remove.
>
> Yep, and thereby cuts the whole gordian knot, throws the
> pieces on the fire and burns them. :-)
>
> Package managers have always seemed to me to be an
> excessively complex solution to a problem that needn't
> have existed in the first place.

Yes!  I love the Zen of the Mac OS X packaging approach.  The best  
install is none at all!  (Of course, I also love apt.)


> I keep hoping that someday Linux will support something
> like MacOSX application bundles and frameworks, but I
> haven't seen any sign of it yet.

We're slowly approaching this level of simplicity in packaging of the  
*source code* of Allmydata.org "Tahoe", using setuptools.

http://allmydata.org/source/tahoe/trunk/docs/install.html

The list of dependencies which are automatically resolved by  
setuptools is visible here: [1].  It currently includes zfec,  
foolscap, simplejson, pycryptopp, nevow, zope.interface, twisted, and  
pywin32.

This automatic resolution of dependencies works while fully  
preserving the user's ability to use their own packages and their own  
packaging tools.  That is:

1. If any of these dependencies are already installed, such as in a  
Debian package or if the user has installed them by hand, then  
installing Tahoe will use the already-installed versions and not  
install anything new, and

2. For any of these dependencies that are not already installed,  
installing Tahoe will *not* write these dependencies into your  
standard system directory (which is potentially a sacred place where  
only your own packaging tool or your root account is allowed to  
tread) but will instead write them into a local, newly-created  
install directory from which you can then run Tahoe.  (This part is  
similar in spirit to the Mac OS packaging technique.)

Also, this install process never downloads anything from the Internet  
at install time, per our policy [2, 3], which also happens to be a  
policy some other people have, e.g. [4, 5].

This works on all of our supported platforms, which includes Linux,  
Solaris, Windows, Cygwin, and Mac OS X.

Oh yes, we also have our buildbot [6] automatically produce Debian  
packages for edgy, feisty, etch, and gutsy.

As far as I know, all of this is accomplished without breaking any of  
the use cases traditionally associated with setuptools /  
easy_install, for example "easy_install allmydata-tahoe" works, and  
if you want "setup.py install" to install into your standard system  
directory, it will.

The reason that I am posting this is to let other programmers know  
that setuptools is actually a pretty useful tool, even if the use  
cases that you want to support are incompatible with certain  
easy_install traditions such as fetching new packages from the  
internet at buildtime or installing into your system directory.

Regards,

Zooko

P.S.  Two days ago I was able to remove twisted from the list of  
"Manual Dependencies" that people have to be aware of in order to try  
out Allmydata Tahoe from the source tarball.

I think I can safely remove pyOpenSSL now, but that remains to be  
properly tested by our buildbot.

I will be able to remove Crypto++ soon, due to the pycryptopp [7]  
library.

If I can figure out a hack to work-around one of the major  
frustrations of setuptools (that you can't simply run "./setup.py  
install --prefix=$FOO"), and if I finish my setuptools plugin to run  
Twisted trial instead pyunit when "./setup.py test", then I'll be  
able to remove GNU make from the dependencies.

That will leave only g++, Python, and OpenSSL as packages that a  
programmer has to manually deal with in order to try out Allmydata  
Tahoe from source.

[1] http://allmydata.org/trac/tahoe/browser/_auto_deps.py
[2] http://allmydata.org/trac/tahoe/wiki/Packaging
[3] http://allmydata.org/trac/tahoe/ticket/229
[4] http://bytes.com/forum/thread666455.html
[5] http://fedoraproject.org/wiki/Packaging/Python/Eggs
[6] http://allmydata.org/buildbot/waterfall?show_events=false
[7] http://allmydata.org/trac/pycryptopp


From nnorwitz at gmail.com  Sat Apr 12 06:34:06 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 11 Apr 2008 21:34:06 -0700
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <47FFDDD0.10900@v.loewis.de>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>
	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>
	<47FF11F6.5010808@v.loewis.de>
	<1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>
	<47FFDDD0.10900@v.loewis.de>
Message-ID: <ee2a432c0804112134s688797bcv2ef8e445aea5891a@mail.gmail.com>

On Fri, Apr 11, 2008 at 2:53 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I just tried it, and although it works, I get this output:
>  > svn: 'post-revprop-change' hook failed; no error output available
>  >
>  > Significant?
>
>  It's the mailer.py hook failing. I'm not quite sure why it fails,
>  and had no time to look it up. It reads
>
>  #!/bin/sh
>  REPOS="$1"
>  REV="$2"
>  USER="$3"
>  PROPNAME="$4"
>
>  /data/repos/projects/hooks/mailer.py propchange "$@"
>
>  The script is the same mailer.py that is also invoked in post-commit.
>  Can anybody see a problem with that?

No, but how many parameters are passed to mailer.py?

  if cmd == 'commit':
    if len(sys.argv) > 5:
      usage()
    if len(sys.argv) > 4:
      config_fname = sys.argv[4]
  elif cmd == 'propchange':
    if len(sys.argv) < 6 or len(sys.argv) > 7:
      usage()
    # ...

For commit, sys.argv must be <= 5, but for propchange sys.argv must ==
6, unless my tired eyes are reading that wrong.  I don't know if the #
of args passed are the same in both cases or not.

n

From asmodai at in-nomine.org  Sat Apr 12 09:32:26 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 12 Apr 2008 09:32:26 +0200
Subject: [Python-Dev] configure on FreeBSD 7
Message-ID: <20080412073226.GL51167@nexus.in-nomine.org>

I do not have this problem on FreeBSD 6.3-STABLE, but on my FreeBSD
7.0-STABLE I get this problem after running an identical ./configure:

[09:11] [asmodai at oni] (0) {0} % make
"Makefile", line 1192: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1194: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1196: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1198: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1200: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1202: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1204: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1206: warning: duplicate script for target "Modules/" ignored
"Makefile", line 1208: warning: duplicate script for target "Modules/" ignored
Graph cycles through thread

make: don't know how to make threadmodule.c. Stop

line 1192 is:

Modules/ thread threadmodule.o: $(srcdir)/Modules/ thread threadmodule.c; $(CC) $(PY_CFLAGS)  -c $(srcdir)/Modules/ thread threadmodule.c -o Modules/ thread threadmodule.o

On my 6.3 box it became:

Modules/threadmodule.o: $(srcdir)/Modules/threadmodule.c; $(CC) $(PY_CFLAGS)  -c $(srcdir)/Modules/threadmodule.c -o Modules/threadmodule.o

As you can see, some spurious 'thread' mentions got inserted.

Apparently this section is made by makedepend.

Has anyone else encountered this?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
No man is good enough to govern another man without the other's consent...

From asmodai at in-nomine.org  Sat Apr 12 11:10:26 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 12 Apr 2008 11:10:26 +0200
Subject: [Python-Dev] configure on FreeBSD 7
In-Reply-To: <20080412073226.GL51167@nexus.in-nomine.org>
References: <20080412073226.GL51167@nexus.in-nomine.org>
Message-ID: <20080412091026.GM51167@nexus.in-nomine.org>

-On [20080412 09:32], Jeroen Ruigrok van der Werven (asmodai at in-nomine.org) wrote:
>Apparently this section is made by makedepend.

I had no updates for makedepend or autotools, but somewhere along the full
upgrade of all my ports makedepend suddenly started to behave, so some
dependency must have caused the previous breakage. I wonder which though,
since I forced a recompile of makedepend before emailing. Peculiar.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
The only trust in this world is that of steel...

From asmodai at in-nomine.org  Sat Apr 12 12:00:27 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 12 Apr 2008 12:00:27 +0200
Subject: [Python-Dev] weird configure (autotools) setup
Message-ID: <20080412100027.GN51167@nexus.in-nomine.org>

Why is CFLAGS in Makefile.pre.in specified as
CFLAGS=         $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
whereas that will negate any CFLAGS you pass to configure?

A normal call to configure (as ./configure --help also explains) can contain
a CFLAGS specification, e.g.:

CFLAGS="-compiler_options" ./configure --config_options

Configure happily compiles and tests everything with the working CFLAGS (as
evident in config.log), but when it comes to the output substitution it
will, of course, not be overridden.

Now, I realize that configure.in talks about BASECFLAGS and OPT, but neither
of these are documented in the ./configure --help output, nor are they
standard when it comes to autotooling. Passing CFLAGS is the defacto
standard.

Furthermore, when passing compiler options to OPT, these are NOT taken along
in the configure tests, which means you might have passed faulty options but
configure will not detect any problems with them. You will only encounter
this when you start building.

So is there any rationale for all this?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
Contentment that derives from knowing when to be content is eternal
contentment...

From martin at v.loewis.de  Sat Apr 12 13:45:08 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 12 Apr 2008 13:45:08 +0200
Subject: [Python-Dev] weird configure (autotools) setup
In-Reply-To: <20080412100027.GN51167@nexus.in-nomine.org>
References: <20080412100027.GN51167@nexus.in-nomine.org>
Message-ID: <4800A0C4.1020601@v.loewis.de>

Jeroen Ruigrok van der Werven wrote:
> Why is CFLAGS in Makefile.pre.in specified as
> CFLAGS=         $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
> whereas that will negate any CFLAGS you pass to configure?

There is a long history to that. The short version is that configure
decides on its own what flags to pass to the compiler; that overrides
any CFLAGS passed by the users. There have been multiple patches over
the years to resolve this, eventually arriving at the status quo.

Regards,
Martin


From fwierzbicki at gmail.com  Sat Apr 12 14:42:00 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Sat, 12 Apr 2008 08:42:00 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
Message-ID: <4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>

On Fri, Apr 11, 2008 at 4:03 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:
> I was also going to suggest a platform independent option.  I like
>  -Xwhat-follows-is-impl-dependent.
This would work just fine for us, and it makes sense to have it
available for all implementations.  If everyone likes this I will
change the patch for CPython accordingly.  How is

-X is reserved for non-standard arguments

-Frank

From ajaksu at gmail.com  Sat Apr 12 15:12:18 2008
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sat, 12 Apr 2008 10:12:18 -0300
Subject: [Python-Dev] Next monthly sprint/bugfix day?
In-Reply-To: <20080410151046.GA11041@isnomore.net>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
	<20080410151046.GA11041@isnomore.net>
Message-ID: <2d75d7660804120612r1b1677bfx74747942c52d8111@mail.gmail.com>

On Thu, Apr 10, 2008 at 12:10 PM, Rodrigo Bernardo Pimentel
<rbp at isnomore.net> wrote:
> On Wed, Apr 09 2008 at 11:12:58AM BRT, Trent Nelson <tnelson at onresolve.com> wrote:
>  > Hi,
>  >
>  > Is there another online sprint/bugfix day in the pipeline?  If not, can
>  > there be? ;-)
>
>  +1.
>
>  The Sao Paulo User's Group gathered to work on the last Bug Day, and I think
>  it was very productive (both for generating patches/comments and for us to
>  get more involved with Python core development).

+1 from me.

Rodrigo: Do you think GruPy-SP could coordinate with python-brasil to
promote local participation in the next Bug Day?

Regards,
Daniel (from Bras?lia-DF)

From rbp at isnomore.net  Sat Apr 12 16:04:32 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Sat, 12 Apr 2008 11:04:32 -0300
Subject: [Python-Dev] Next monthly sprint/bugfix day?
In-Reply-To: <2d75d7660804120612r1b1677bfx74747942c52d8111@mail.gmail.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
	<20080410151046.GA11041@isnomore.net>
	<2d75d7660804120612r1b1677bfx74747942c52d8111@mail.gmail.com>
Message-ID: <20080412140431.GB10294@isnomore.net>

On Sat, Apr 12 2008 at 10:12:18AM BRT, "Daniel (ajax) Diniz" <ajaksu at gmail.com> wrote:
> On Thu, Apr 10, 2008 at 12:10 PM, Rodrigo Bernardo Pimentel
> <rbp at isnomore.net> wrote:
> > On Wed, Apr 09 2008 at 11:12:58AM BRT, Trent Nelson <tnelson at onresolve.com> wrote:
> >  > Hi,
> >  >
> >  > Is there another online sprint/bugfix day in the pipeline?  If not, can
> >  > there be? ;-)
> >
> >  +1.
> >
> >  The Sao Paulo User's Group gathered to work on the last Bug Day, and I think
> >  it was very productive (both for generating patches/comments and for us to
> >  get more involved with Python core development).
> 
> +1 from me.
> 
> Rodrigo: Do you think GruPy-SP could coordinate with python-brasil to
> promote local participation in the next Bug Day?

Sure. When the next Bug Day is confirmed, we'll take it to python-brasil and
the local user's groups.


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From brett at python.org  Sat Apr 12 18:12:19 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 12 Apr 2008 18:12:19 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
Message-ID: <bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>

On Sat, Apr 12, 2008 at 5:42 AM, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
> On Fri, Apr 11, 2008 at 4:03 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:
>  > I was also going to suggest a platform independent option.  I like
>  >  -Xwhat-follows-is-impl-dependent.
>  This would work just fine for us, and it makes sense to have it
>  available for all implementations.  If everyone likes this I will
>  change the patch for CPython accordingly.  How is
>
>  -X is reserved for non-standard arguments
>

Fine by me.

-Brett

From lists at cheimes.de  Sat Apr 12 18:39:01 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 12 Apr 2008 18:39:01 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>
Message-ID: <4800E5A5.4040600@cheimes.de>

Brett Cannon schrieb:
>>  -X is reserved for non-standard arguments
> 
> Fine by me.

And implemented in r62293 (trunk)

Christian


From martin at v.loewis.de  Sat Apr 12 19:34:58 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 12 Apr 2008 19:34:58 +0200
Subject: [Python-Dev] generated NEWS files
In-Reply-To: <ee2a432c0804112134s688797bcv2ef8e445aea5891a@mail.gmail.com>
References: <1afaf6160804101642la035a44k65625bda0ca893e5@mail.gmail.com>	<ca471dc20804101755o5e56a413uadd30115d472562e@mail.gmail.com>	<47FF11F6.5010808@v.loewis.de>	<1afaf6160804111428m24e9e94fv25d56f9bbb994b6a@mail.gmail.com>	<47FFDDD0.10900@v.loewis.de>
	<ee2a432c0804112134s688797bcv2ef8e445aea5891a@mail.gmail.com>
Message-ID: <4800F2C2.30709@v.loewis.de>

>>  #!/bin/sh
>>  REPOS="$1"
>>  REV="$2"
>>  USER="$3"
>>  PROPNAME="$4"
>>
>>  /data/repos/projects/hooks/mailer.py propchange "$@"
>>
>>  The script is the same mailer.py that is also invoked in post-commit.
>>  Can anybody see a problem with that?
> 
> No, but how many parameters are passed to mailer.py?
> 
>   if cmd == 'commit':
>     if len(sys.argv) > 5:
>       usage()
>     if len(sys.argv) > 4:
>       config_fname = sys.argv[4]
>   elif cmd == 'propchange':
>     if len(sys.argv) < 6 or len(sys.argv) > 7:
>       usage()
>     # ...
> 
> For commit, sys.argv must be <= 5, but for propchange sys.argv must ==
> 6, unless my tired eyes are reading that wrong.  I don't know if the #
> of args passed are the same in both cases or not.

I've now looked into this; in Subversion 1.2, the number of arguments
passed to post-revprop-change had changed, giving an addition "action"
as $5. This broke the old mailer.py, which expected only 4 arguments.

The rationale for this additional argument was to be able to tell
whether a property had been added, modified, or deleted; if it was
added, the old value was supplied as stdin (since revprops themselves
aren't versioned).

I've now updated mailer.py, merging our (few) changes into it.
If anything seems broken now, please let me know.

Regards,
Martin

From skip at pobox.com  Sat Apr 12 20:02:02 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 12 Apr 2008 13:02:02 -0500
Subject: [Python-Dev] Python 2.4.4/2.4.5 test_pty failure on Solaris 10
Message-ID: <18432.63770.323783.565994@montanaro-dyndns-org.local>


I know this is old stuff, but...

I want to update our Python 2.4 installation at work from 2.4.2 to 2.4.5
(the latest 2.4 source release).  I get a test failure for test_pty, an
extra ^M at the end of one line.  I don't get a failure in the 2.4.2
installation, but the 2.4.4 and 2.4.5 both fail this test.  Looking at the
code in test_pty.py, it appears to me that r43570 fixed things for OSF/1 and
IRIX which both do weird things with output while breaking things for any
other platform by suppressing the \r\n -> \n mapping which used to be
performed for all platforms.  So, for Solaris, that mapping doesn't happen
and the actual and expected outputs don't agree.

I'm not suggesting this needs to be fixed, I'm just looking for confirmation
of my hypothesis if someone has a moment to compare the two revisions:

    http://svn.python.org/view/python/branches/release24-maint/Lib/test/test_pty.py?rev=43570&r1=42233&r2=43570

Thanks,

Skip

From asmodai at in-nomine.org  Sat Apr 12 20:09:36 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 12 Apr 2008 20:09:36 +0200
Subject: [Python-Dev] Python 2.6a2 execution times with various	compilers
In-Reply-To: <20080411151024.GJ51167@nexus.in-nomine.org>
References: <20080411151024.GJ51167@nexus.in-nomine.org>
Message-ID: <20080412180936.GP51167@nexus.in-nomine.org>

I did some more tests concentrating on GCC, partly based on the feedback I
got, results at
http://www.in-nomine.org/2008/04/12/python-26-compiler-options-results/

Executive summary: Python needs to be compiled with -O2 or -O3. Not doing
so, no optimization level, results with GCC 4.2.1 in a doubling of execution
time. Using just -O1 is still ~15% slower than using -O2.

Using -mtune=native -march=native can shave of 0,1/0,2 seconds, but
otherwise I did not find much difference in using having march or mfpmath
present.

Profile-guided optimization did not help much, as might be expected, it
pushed about the same kind of optimization as the mtune/march combination.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
All know that the drop merges into the ocean but few know that the ocean
merges into the drop.

From fwierzbicki at gmail.com  Sat Apr 12 20:27:57 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Sat, 12 Apr 2008 14:27:57 -0400
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4800E5A5.4040600@cheimes.de>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>
	<4800E5A5.4040600@cheimes.de>
Message-ID: <4dab5f760804121127l3713a329v1927519864b33d6b@mail.gmail.com>

On Sat, Apr 12, 2008 at 12:39 PM, Christian Heimes <lists at cheimes.de> wrote:
> Brett Cannon schrieb:
>
> >>  -X is reserved for non-standard arguments
>  >
>  > Fine by me.
>
>  And implemented in r62293 (trunk)
Great, thanks!  While I'd love to have *both* -X and -J, is that okay
with the other devs?

-Frank

From martin at v.loewis.de  Sat Apr 12 22:08:31 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 12 Apr 2008 22:08:31 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804121127l3713a329v1927519864b33d6b@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>	<bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>	<4800E5A5.4040600@cheimes.de>
	<4dab5f760804121127l3713a329v1927519864b33d6b@mail.gmail.com>
Message-ID: <480116BF.2040208@v.loewis.de>

> Great, thanks!  While I'd love to have *both* -X and -J, is that okay
> with the other devs?

+0. If we ever run out of letters for command line options to have
to collect -J, we have deeper problems than having to coordinate
with Jython whether the letter is still available.

Regards,
Martin

From greg at krypto.org  Sun Apr 13 00:01:32 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 12 Apr 2008 15:01:32 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
Message-ID: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>

http://bugs.python.org/issue1481036

Basically as things are now EOFError is on its own but often wants to be
handled the same as other I/O errors that EnvironmentError currently covers.

Many uses of EOFError in our code base do not provide it any arguments so it
doesn't really fit the (errno, message [, filename]) tuple style that
EnvironmentError promises.  But we could fudge that with a reasonable
default (whats reasonable?) if we rerooted this under EnvironmentError.

Alternatively the bug suggests a new parent exception for EnvironmentError
and EOFError both to inherit from.

Last time changing the heirarchy around this came up there was pushback
against adding yet another exception type so I'm thinking the simple
re-rooting may be the best answer if anything is done at all.

any thoughts?

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080412/569d5180/attachment.htm 

From greg at krypto.org  Sun Apr 13 00:21:32 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 12 Apr 2008 15:21:32 -0700
Subject: [Python-Dev] Python 2.6a2 execution times with various compilers
In-Reply-To: <20080412180936.GP51167@nexus.in-nomine.org>
References: <20080411151024.GJ51167@nexus.in-nomine.org>
	<20080412180936.GP51167@nexus.in-nomine.org>
Message-ID: <52dc1c820804121521u4c8f69d8q1bf0d0bb9b00e03a@mail.gmail.com>

On Sat, Apr 12, 2008 at 11:09 AM, Jeroen Ruigrok van der Werven <
asmodai at in-nomine.org> wrote:

> I did some more tests concentrating on GCC, partly based on the feedback I
> got, results at
> http://www.in-nomine.org/2008/04/12/python-26-compiler-options-results/
>
> Executive summary: Python needs to be compiled with -O2 or -O3. Not doing
> so, no optimization level, results with GCC 4.2.1 in a doubling of
> execution
> time. Using just -O1 is still ~15% slower than using -O2.
>
> Using -mtune=native -march=native can shave of 0,1/0,2 seconds, but
> otherwise I did not find much difference in using having march or mfpmath
> present.
>
> Profile-guided optimization did not help much, as might be expected, it
> pushed about the same kind of optimization as the mtune/march combination.
>

With gcc 4.1.3 i'm finding that profile guided optimization when trained on
pybench or regrtest does make a measurable difference (2-5% overall time
with 10-20% on some pybench tests).  I haven't run benchmarks enough times
to be confident in my results yet, I'll report back with data once I have
it.  I'm testing both pybench and regrtest as profiling training runs.

I will check in a special makefile target for easy gcc profile guided
compiles shortly so that those who want faster builds easily produce them.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080412/6bfe80c4/attachment.htm 

From brett at python.org  Sun Apr 13 00:52:42 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 12 Apr 2008 15:52:42 -0700
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <480116BF.2040208@v.loewis.de>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>
	<4800E5A5.4040600@cheimes.de>
	<4dab5f760804121127l3713a329v1927519864b33d6b@mail.gmail.com>
	<480116BF.2040208@v.loewis.de>
Message-ID: <bbaeab100804121552pba30e4j392d23b13f14db24@mail.gmail.com>

On Sat, Apr 12, 2008 at 1:08 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Great, thanks!  While I'd love to have *both* -X and -J, is that okay
>  > with the other devs?
>
>  +0. If we ever run out of letters for command line options to have
>  to collect -J, we have deeper problems than having to coordinate
>  with Jython whether the letter is still available.

+0

-Brett

From stephen at xemacs.org  Sun Apr 13 01:14:14 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 13 Apr 2008 08:14:14 +0900
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
Message-ID: <87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>

Frank Wierzbicki writes:
 > On Fri, Apr 11, 2008 at 4:03 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:
 > > I was also going to suggest a platform independent option.  I like
 > >  -Xwhat-follows-is-impl-dependent.
 > This would work just fine for us, and it makes sense to have it
 > available for all implementations.  If everyone likes this I will
 > change the patch for CPython accordingly.  How is
 > 
 > -X is reserved for non-standard arguments

How about "-X is reserved for implementation-specific arguments"?  Ie,
I suppose that the intent is not that these arguments won't be
standardized, it's that they be standardized by the affected
implementations.


From martin at v.loewis.de  Sun Apr 13 07:27:48 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2008 07:27:48 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <480199D4.20701@v.loewis.de>

> How about "-X is reserved for implementation-specific arguments"?  Ie,
> I suppose that the intent is not that these arguments won't be
> standardized, it's that they be standardized by the affected
> implementations.

Isn't that bikeshedding?

Regards,
Martin

From stephen at xemacs.org  Sun Apr 13 09:00:21 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 13 Apr 2008 16:00:21 +0900
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <480199D4.20701@v.loewis.de>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480199D4.20701@v.loewis.de>
Message-ID: <873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > > How about "-X is reserved for implementation-specific arguments"?

 > Isn't that bikeshedding?

No.  I think "implementation-specific" is definitely more accurate,
and I was hoping the suggestion might get an immediate "good idea,
implemented", from somebody already looking at that code.

From martin at v.loewis.de  Sun Apr 13 10:23:59 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2008 10:23:59 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>	<480199D4.20701@v.loewis.de>
	<873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4801C31F.1030300@v.loewis.de>

>  > > How about "-X is reserved for implementation-specific arguments"?
> 
>  > Isn't that bikeshedding?
> 
> No.  I think "implementation-specific" is definitely more accurate,
> and I was hoping the suggestion might get an immediate "good idea,
> implemented", from somebody already looking at that code.

It's already committed, so one would have to go back and change it.

Regards,
Martin


From asmodai at in-nomine.org  Sun Apr 13 10:25:03 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sun, 13 Apr 2008 10:25:03 +0200
Subject: [Python-Dev] Python 2.6a2 execution times with various	compilers
In-Reply-To: <52dc1c820804121521u4c8f69d8q1bf0d0bb9b00e03a@mail.gmail.com>
References: <20080411151024.GJ51167@nexus.in-nomine.org>
	<20080412180936.GP51167@nexus.in-nomine.org>
	<52dc1c820804121521u4c8f69d8q1bf0d0bb9b00e03a@mail.gmail.com>
Message-ID: <20080413082503.GQ51167@nexus.in-nomine.org>

-On [20080413 00:47], Gregory P. Smith (greg at krypto.org) wrote:
>With gcc 4.1.3 i'm finding that profile guided optimization when trained on
>pybench or regrtest does make a measurable difference (2-5% overall time with
>10-20% on some pybench tests).  I haven't run benchmarks enough times to be
>confident in my results yet, I'll report back with data once I have it.  I'm
>testing both pybench and regrtest as profiling training runs.

It seems GCC 4.2.4 yields worse code for Python with the same options as
4.2.1, I measured about ~7%-8% slowdown (~0,5 seconds) on my test.

Granted, in general this might all be nitpicking, but for our friends in the
calculating departments this might be quite useful to know. The differences
are in general not concentrated in specific sections of pybench, but are
uniformly distributed. I know my employer can use such additional free
optimizations since our jobs spawn in many hours of execution. Next to
optimizing the source code, of course, this will also shave off quite a lot
of execution time.

>I will check in a special makefile target for easy gcc profile guided compiles
>shortly so that those who want faster builds easily produce them.

That would be interesting I think.
I went with -fprofile-generate and -fprofile-use in my small test.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
Put your heart, mind, intellect and soul even to your smallest acts. This
is the secret of success...

From turnbull at sk.tsukuba.ac.jp  Sun Apr 13 19:28:30 2008
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 14 Apr 2008 02:28:30 +0900
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <4801C31F.1030300@v.loewis.de>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>
	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>
	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>
	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480199D4.20701@v.loewis.de>
	<873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4801C31F.1030300@v.loewis.de>
Message-ID: <87zlrxfyv5.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > >  > > How about "-X is reserved for implementation-specific arguments"?
 > > 
 > >  > Isn't that bikeshedding?
 > > 
 > > No.  I think "implementation-specific" is definitely more accurate,
 > > and I was hoping the suggestion might get an immediate "good idea,
 > > implemented", from somebody already looking at that code.
 > 
 > It's already committed, so one would have to go back and change it.

Now, *you* are bikeshedding.

I know that it was committed, so I politely made the suggestion,
half-expecting it to get dropped on the floor.  If so, no great loss,
and I wouldn't have followed up (except maybe with a patch on the
tracker).

Is that discouraged on this list?  (That's a real question.)

From martin at v.loewis.de  Sun Apr 13 19:59:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Apr 2008 19:59:30 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <87zlrxfyv5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<ca471dc20804111040t5a6f5521h1bfb6267c29209ad@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>	<480199D4.20701@v.loewis.de>	<873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>	<4801C31F.1030300@v.loewis.de>
	<87zlrxfyv5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <48024A02.5070904@v.loewis.de>

>  > >  > Isn't that bikeshedding?
>  > > 
>  > > No.  I think "implementation-specific" is definitely more accurate,
>  > > and I was hoping the suggestion might get an immediate "good idea,
>  > > implemented", from somebody already looking at that code.
>  > 
>  > It's already committed, so one would have to go back and change it.
> 
> Now, *you* are bikeshedding.
> 
> I know that it was committed, so I politely made the suggestion,
> half-expecting it to get dropped on the floor.  If so, no great loss,
> and I wouldn't have followed up (except maybe with a patch on the
> tracker).
> 
> Is that discouraged on this list?  (That's a real question.)

Making suggestions on the list, and then following up with patch,
is certainly encouraged, and happens all the time.

I just think that *this* specific proposed change is more effort to
talk about than it's worth. It may be more accurate, but changing the
phrasing of the message that you get when you invoke a command line
option that you shouldn't invoke in the first place, and can't know
about unless you've been following this thread or read the source ...
is such a minor thing that accuracy doesn't really matter, and is
primarily a matter of personal taste.

Regards,
Martin


From lists at cheimes.de  Sun Apr 13 20:10:18 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 13 Apr 2008 20:10:18 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <bbaeab100804121552pba30e4j392d23b13f14db24@mail.gmail.com>
References: <4dab5f760804111038u48617a40k20a376213216ea79@mail.gmail.com>	<bbaeab100804111151p6f5a52eck57c36a132e847795@mail.gmail.com>	<4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>	<bbaeab100804120912n32f0b65ag1c9c05eb2fa3d65c@mail.gmail.com>	<4800E5A5.4040600@cheimes.de>	<4dab5f760804121127l3713a329v1927519864b33d6b@mail.gmail.com>	<480116BF.2040208@v.loewis.de>
	<bbaeab100804121552pba30e4j392d23b13f14db24@mail.gmail.com>
Message-ID: <fttia9$rjh$1@ger.gmane.org>

Brett Cannon schrieb:
>>  +0. If we ever run out of letters for command line options to have
>>  to collect -J, we have deeper problems than having to coordinate
>>  with Jython whether the letter is still available.
> 
> +0


Shall I remove the reservation of -J again?

Christian


From asmodai at in-nomine.org  Sun Apr 13 21:27:20 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sun, 13 Apr 2008 21:27:20 +0200
Subject: [Python-Dev] Reserving an arg space for Jython
In-Reply-To: <48024A02.5070904@v.loewis.de>
References: <4dab5f760804111213o56ce451cxc7315b5fa088f3@mail.gmail.com>
	<7AD436E4270DD54A94238001769C2227011DAFFE40CE@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0804111303i2cfe6655pbcbed7ac253f11fe@mail.gmail.com>
	<4dab5f760804120542oeae7e7ek2ab84bbfb0c8223c@mail.gmail.com>
	<87abjyis3d.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480199D4.20701@v.loewis.de>
	<873apqgry2.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4801C31F.1030300@v.loewis.de>
	<87zlrxfyv5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48024A02.5070904@v.loewis.de>
Message-ID: <20080413192720.GV51167@nexus.in-nomine.org>

-On [20080413 19:59], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>Making suggestions on the list, and then following up with patch,
>is certainly encouraged, and happens all the time.

Here you go.

>I just think that *this* specific proposed change is more effort to
>talk about than it's worth.

So I wonder why it couldn't have been changed then instead of spending
another email on the subject. :P <tongue firmly planted in cheek>

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
Without you, I am nothing...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: X.diff
Type: text/x-diff
Size: 736 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20080413/d2f87a49/attachment.diff 

From hrvoje.niksic at avl.com  Mon Apr 14 15:56:47 2008
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Mon, 14 Apr 2008 15:56:47 +0200
Subject: [Python-Dev] Pickle format machine independence
Message-ID: <1208181407.28170.80.camel@localhost>

Are pickle files intended to be readable across different machine
architectures?  The documentation states unequivocally that they are
compatible across Python versions, but compatibility across machine
architectures (wrt to differences in size and layout of primitive C
types) is not explicitly addressed.

One example where I stumbled upon the incompatibility is the pickling of
arrays.  While pickle is normally very careful to write out numbers in a
platform-independent way, arrays are written out in "tostring" format.
This is filed under http://bugs.python.org/issue2389.

I can work around this issue in my application, but if this is
considered a bug, I'd prefer to fix it in Python instead.



From schmir at gmail.com  Mon Apr 14 18:12:35 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 14 Apr 2008 18:12:35 +0200
Subject: [Python-Dev] very bad network performance
Message-ID: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>

Hi all,

I'm using mercurial with the release25-maint branch. I noticed that checking
out a local repository now takes more than
5 minutes (it should be around 30s).

I've tracked it down to this change:
http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
this is svn revision 61009. Here is the diff inline:

--- a/Lib/socket.py	Fri Mar 23 14:27:29 2007 +0100
+++ b/Lib/socket.py	Sat Feb 23 20:30:59 2008 +0100
@@ -305,7 +305,7 @@
             self._rbuf = ""
             while True:
                 left = size - buf_len
-                recv_size = max(self._rbufsize, left)
+                recv_size = min(self._rbufsize, left)
                 data = self._sock.recv(recv_size)
                 if not data:
                     break



self._rbufsize if 1, and so the code reads one byte at a time. this is
clearly wrong, I'm posting it to the mailing list, as I don't want
this issue to get lost in the bugtracker.


- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080414/eeff8dfc/attachment.htm 

From tjreedy at udel.edu  Mon Apr 14 19:25:27 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 14 Apr 2008 13:25:27 -0400
Subject: [Python-Dev] very bad network performance
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
Message-ID: <fu0425$l1q$1@ger.gmane.org>


"Ralf Schmitt" <schmir at gmail.com> wrote in message 
news:932f8baf0804140912u54adc7d5md7261541857f21bd at mail.gmail.com...
| Hi all,
|
| I'm using mercurial with the release25-maint branch. I noticed that 
checking
| out a local repository now takes more than
| 5 minutes (it should be around 30s).
|
| I've tracked it down to this change:
| http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
| this is svn revision 61009. Here is the diff inline:
|
| --- a/Lib/socket.py Fri Mar 23 14:27:29 2007 +0100
| +++ b/Lib/socket.py Sat Feb 23 20:30:59 2008 +0100
| @@ -305,7 +305,7 @@
|             self._rbuf = ""
|             while True:
|                 left = size - buf_len
| -                recv_size = max(self._rbufsize, left)
| +                recv_size = min(self._rbufsize, left)
|                 data = self._sock.recv(recv_size)
|                 if not data:
|                     break
|
|
|
| self._rbufsize if 1, and so the code reads one byte at a time. this is
| clearly wrong, I'm posting it to the mailing list, as I don't want
| this issue to get lost in the bugtracker.

--------------------------------------------------------------------------------

It is at least as likely to get lost here.  There is a mailing list for new 
tracker items that many devs subscribe to.




From guido at python.org  Mon Apr 14 19:56:17 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 10:56:17 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <fu0425$l1q$1@ger.gmane.org>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<fu0425$l1q$1@ger.gmane.org>
Message-ID: <ca471dc20804141056v1f275echc18c60b03abfa0d5@mail.gmail.com>

Ralf,

Terry is right. Please file a bug. I do think there may be a problem
with that change but I don't have the time to review it in depth.
Hopefully others will. I do recall that sockets reading one byte at a
time has been a problem before -- I recall a bug about this in the
1.5.2 era for Windows... Too bad it's back. :-(

--Guido

On Mon, Apr 14, 2008 at 10:25 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>  "Ralf Schmitt" <schmir at gmail.com> wrote in message
>  news:932f8baf0804140912u54adc7d5md7261541857f21bd at mail.gmail.com...
>
>
> | Hi all,
>  |
>  | I'm using mercurial with the release25-maint branch. I noticed that
>  checking
>  | out a local repository now takes more than
>  | 5 minutes (it should be around 30s).
>  |
>  | I've tracked it down to this change:
>  | http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
>  | this is svn revision 61009. Here is the diff inline:
>  |
>  | --- a/Lib/socket.py Fri Mar 23 14:27:29 2007 +0100
>  | +++ b/Lib/socket.py Sat Feb 23 20:30:59 2008 +0100
>  | @@ -305,7 +305,7 @@
>  |             self._rbuf = ""
>  |             while True:
>  |                 left = size - buf_len
>  | -                recv_size = max(self._rbufsize, left)
>  | +                recv_size = min(self._rbufsize, left)
>  |                 data = self._sock.recv(recv_size)
>  |                 if not data:
>  |                     break
>  |
>  |
>  |
>  | self._rbufsize if 1, and so the code reads one byte at a time. this is
>  | clearly wrong, I'm posting it to the mailing list, as I don't want
>  | this issue to get lost in the bugtracker.
>
>  --------------------------------------------------------------------------------
>
>  It is at least as likely to get lost here.  There is a mailing list for new
>  tracker items that many devs subscribe to.
>
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 14 19:59:51 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 10:59:51 -0700
Subject: [Python-Dev] Pickle format machine independence
In-Reply-To: <1208181407.28170.80.camel@localhost>
References: <1208181407.28170.80.camel@localhost>
Message-ID: <ca471dc20804141059k4d5c73e3s9db10d7328f8b0c9@mail.gmail.com>

On Mon, Apr 14, 2008 at 6:56 AM, Hrvoje Nik?i? <hrvoje.niksic at avl.com> wrote:
> Are pickle files intended to be readable across different machine
>  architectures?  The documentation states unequivocally that they are
>  compatible across Python versions, but compatibility across machine
>  architectures (wrt to differences in size and layout of primitive C
>  types) is not explicitly addressed.

They're supposed to be compatible across all architectures.

>  One example where I stumbled upon the incompatibility is the pickling of
>  arrays.  While pickle is normally very careful to write out numbers in a
>  platform-independent way, arrays are written out in "tostring" format.
>  This is filed under http://bugs.python.org/issue2389.
>
>  I can work around this issue in my application, but if this is
>  considered a bug, I'd prefer to fix it in Python instead.

It may not be easy to fix this in a backwards-compatible way, but I
agree that there's something fishy there. If you can think of a fix,
please do submit a patch!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From curt at hagenlocher.org  Mon Apr 14 20:10:12 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 14 Apr 2008 11:10:12 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
Message-ID: <d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>

On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>
> I've tracked it down to this change:
> http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
> this is svn revision 61009.
> [...]
> self._rbufsize if 1, and so the code reads one byte at a time

The change is correct, but exposes a flaw earlier in the same method.
"_rbufsize == 1" represents a request to buffer "by line", which is
clearly irrelevant in this context.  A request to read n bytes should
just use the default buffer size if buffering "by line".  Sample patch
is attached.


--
Curt Hagenlocher
curt at hagenlocher.org
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: socket.py.diff
Url: http://mail.python.org/pipermail/python-dev/attachments/20080414/4003efd8/attachment.txt 

From guido at python.org  Mon Apr 14 20:22:05 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 11:22:05 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
Message-ID: <ca471dc20804141122g20aededu66447acc677ed439@mail.gmail.com>

Eek! Please use the bug tracker.

On Mon, Apr 14, 2008 at 11:10 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>  >
>  > I've tracked it down to this change:
>  > http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
>  > this is svn revision 61009.
>  > [...]
>
> > self._rbufsize if 1, and so the code reads one byte at a time
>
>  The change is correct, but exposes a flaw earlier in the same method.
>  "_rbufsize == 1" represents a request to buffer "by line", which is
>  clearly irrelevant in this context.  A request to read n bytes should
>  just use the default buffer size if buffering "by line".  Sample patch
>  is attached.
>
>
>  --
>  Curt Hagenlocher
>  curt at hagenlocher.org
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Mon Apr 14 20:36:33 2008
From: janssen at parc.com (Bill Janssen)
Date: Mon, 14 Apr 2008 11:36:33 PDT
Subject: [Python-Dev] very bad network performance
In-Reply-To: <ca471dc20804141056v1f275echc18c60b03abfa0d5@mail.gmail.com> 
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<fu0425$l1q$1@ger.gmane.org>
	<ca471dc20804141056v1f275echc18c60b03abfa0d5@mail.gmail.com>
Message-ID: <08Apr14.113640pdt."58696"@synergy1.parc.xerox.com>

There's some really convoluted code in socket._fileobject.__init__()
here.  When initializing a _fileobject, if the 'bufsize' parameter is
explicitly given as zero, that's turned into an _rbufsize of 1, which,
combined with the 'min' change, will produce the read-one-byte
behavior.  The code for setting _rbufsize seems odd; be nice if it was
commented with some notes on why these specific selections were made.

        if bufsize < 0:
            bufsize = self.default_bufsize
        if bufsize == 0:
            self._rbufsize = 1
        elif bufsize == 1:
            self._rbufsize = self.default_bufsize
        else:
            self._rbufsize = bufsize
        self._wbufsize = bufsize

It also depends on whether 'read' is called with an explicit # of
bytes to read (which appears to be the case here).

So, it's not the code in socket.py, necessarily; it's the code which
opens the socket, most likely.  The only library which seems to use a
bufsize of zero is httplib (which has a lot of other problems as
well).  I think the change cited below (while IMO correct) will affect
a number of other HTTP-based services, as well.

Bill

> Ralf,
> 
> Terry is right. Please file a bug. I do think there may be a problem
> with that change but I don't have the time to review it in depth.
> Hopefully others will. I do recall that sockets reading one byte at a
> time has been a problem before -- I recall a bug about this in the
> 1.5.2 era for Windows... Too bad it's back. :-(
> 
> --Guido
> 
> On Mon, Apr 14, 2008 at 10:25 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> >
> >  "Ralf Schmitt" <schmir at gmail.com> wrote in message
> >  news:932f8baf0804140912u54adc7d5md7261541857f21bd at mail.gmail.com...
> >
> >
> > | Hi all,
> >  |
> >  | I'm using mercurial with the release25-maint branch. I noticed that
> >  checking
> >  | out a local repository now takes more than
> >  | 5 minutes (it should be around 30s).
> >  |
> >  | I've tracked it down to this change:
> >  | http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
> >  | this is svn revision 61009. Here is the diff inline:
> >  |
> >  | --- a/Lib/socket.py Fri Mar 23 14:27:29 2007 +0100
> >  | +++ b/Lib/socket.py Sat Feb 23 20:30:59 2008 +0100
> >  | @@ -305,7 +305,7 @@
> >  |             self._rbuf = ""
> >  |             while True:
> >  |                 left = size - buf_len
> >  | -                recv_size = max(self._rbufsize, left)
> >  | +                recv_size = min(self._rbufsize, left)
> >  |                 data = self._sock.recv(recv_size)
> >  |                 if not data:
> >  |                     break
> >  |
> >  |
> >  |
> >  | self._rbufsize if 1, and so the code reads one byte at a time. this is
> >  | clearly wrong, I'm posting it to the mailing list, as I don't want
> >  | this issue to get lost in the bugtracker.
> >
> >  --------------------------------------------------------------------------------
> >
> >  It is at least as likely to get lost here.  There is a mailing list for new
> >  tracker items that many devs subscribe to.






From guido at python.org  Mon Apr 14 21:12:27 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 12:12:27 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
In-Reply-To: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
Message-ID: <ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>

Offhand, -0. I don't think of EOFError as an environmental error. Its
primary purpose was to have something raised by raw_input() (in 3.0,
input()) when there is no more input. This is quite a different level
of error than what EnvironmentError typically means (a problem in the
filesystem or network, or a permissions thing).

On Sat, Apr 12, 2008 at 3:01 PM, Gregory P. Smith <greg at krypto.org> wrote:
> http://bugs.python.org/issue1481036
>
> Basically as things are now EOFError is on its own but often wants to be
> handled the same as other I/O errors that EnvironmentError currently covers.
>
> Many uses of EOFError in our code base do not provide it any arguments so it
> doesn't really fit the (errno, message [, filename]) tuple style that
> EnvironmentError promises.  But we could fudge that with a reasonable
> default (whats reasonable?) if we rerooted this under EnvironmentError.
>
> Alternatively the bug suggests a new parent exception for EnvironmentError
> and EOFError both to inherit from.
>
> Last time changing the heirarchy around this came up there was pushback
> against adding yet another exception type so I'm thinking the simple
> re-rooting may be the best answer if anything is done at all.
>
> any thoughts?
>
> -gps
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amk at amk.ca  Mon Apr 14 21:17:25 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 14 Apr 2008 15:17:25 -0400
Subject: [Python-Dev] very bad network performance
In-Reply-To: <d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
Message-ID: <20080414191725.GA23517@amk-desktop.matrixgroup.net>

On Mon, Apr 14, 2008 at 11:10:12AM -0700, Curt Hagenlocher wrote:
>               while True:
>                   left = size - buf_len
> !                 recv_size = max(self._rbufsize, left)
>                   data = self._sock.recv(recv_size)

What version is this patch against?  (The last 2.5 release, maybe?)
The max() in the above line should be min(), because you want to use
the *smaller* number of the buffer size and the # of remaining bytes
to read, not the *larger*.  This code is using min() in both 25-maint
and trunk.

--amk


From curt at hagenlocher.org  Mon Apr 14 21:34:35 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 14 Apr 2008 12:34:35 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <20080414191725.GA23517@amk-desktop.matrixgroup.net>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<20080414191725.GA23517@amk-desktop.matrixgroup.net>
Message-ID: <d2155e360804141234u6d0f8b20w347b34d5ef68ff61@mail.gmail.com>

On Mon, Apr 14, 2008 at 12:17 PM, A.M. Kuchling <amk at amk.ca> wrote:
> On Mon, Apr 14, 2008 at 11:10:12AM -0700, Curt Hagenlocher wrote:
> >               while True:
> >                   left = size - buf_len
> > !                 recv_size = max(self._rbufsize, left)
> >                   data = self._sock.recv(recv_size)
>
> What version is this patch against?  (The last 2.5 release, maybe?)

Yes, sorry.  I thought I had checked this against the repository --
particularly because the max->min change is what kicked off this
thread.

--
Curt Hagenlocher
curt at hagenlocher.org

From bonelake at gmail.com  Mon Apr 14 21:45:14 2008
From: bonelake at gmail.com (Brad Miller)
Date: Mon, 14 Apr 2008 14:45:14 -0500
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
Message-ID: <6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>

After posting a patch to implement this some good discussion followed   
see:  http://bugs.python.org/issue2610

It was suggested that a broader discussion might be in order around  
the issue of iterators and how they are displayed in the command line  
interpreter.
Several new iterators have appeared in Python 3.0 that makes the  
language less transparent to beginning programmers.  The examples that  
immediately come to mind are shown below, and I would guess there are  
others I haven't run across yet.


 >>> range(10)
range(0, 10)
 >>> myd = {chr(i):i for i in range(32,42)}
 >>> myd.keys()
<dict_keys object at 0xf31f0>
 >>> myd.values()
<dict_values object at 0xf3220>
 >>> myd.items()
<dict_items object at 0xf31f0>


Although none of the above are a problem for intermediate or advanced  
programmers  I would like to find a way so that beginning students  
would automatically get a more helpful representation when they  
evaluate expressions in the interpreter.

My solution of implementing the __str__ method for range is one  
solution, and that could also be done for the dict_xxx objects as  
well.  Other solutions that were suggested were to include some kind  
of a module that overrides sys.displayhook or to simply make the  
command line interpreter more intelligence.  For example it already  
handles a return value of None in a special way, maybe it should do  
something for these iterators as well.

Any other comments or ideas?

Thanks,

Brad

On Apr 7, 2008, at 6:24 PM, Guido van Rossum wrote:

> I'd object to it returning something that resembles a list too
> closely, but I could live with str(range(3)) return <0, 1, 2>. We
> should probably have a cutoff so that if there are more than 6 values
> it'll show the first 3 values, then dots, then the last 2 values. (The
> cutoff would be computed so that '...' always represents at least 2
> values.
>
> On Mon, Apr 7, 2008 at 4:14 PM, Brad Miller <bonelake at gmail.com>  
> wrote:
>> Hi,
>>
>> I use Python in my CS1 and CS2 curriculum and I have a question.
>> As I've been using the Python 3.0 alphas one of the things that I am
>> bothered by is that I cannot see the sequence produced by range
>> without introducing students to the list() function.
>>
>> I typically introduce range on day 1 of class and show students what
>> it produces without making a big deal out of the fact that it creates
>> a list.  They all accept this and things work out nicely when I
>> introduce lists for real in a week or two.
>>
>> My question is why couldn't the __str__ method for the range object  
>> be
>> more friendly and show a representation of the sequence?  I  
>> understand
>> why __repr__ should return range(0,10) for an object created using
>> range(10) but couldn't print(range(10)) produce [0, 1, 2, ... 9]
>> The ... could even be used if the sequence were excessively wrong.
>>
>> If this is acceptable, I would be happy to accept the challenge of
>> providing a patch.
>>
>> Thanks,
>>
>> Brad
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080414/be28e179/attachment.htm 

From lists at cheimes.de  Mon Apr 14 23:05:57 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 14 Apr 2008 23:05:57 +0200
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <20080414205105.B45C11E4005@bag.python.org>
References: <20080414205105.B45C11E4005@bag.python.org>
Message-ID: <4803C735.10100@cheimes.de>

alexandre.vassalotti schrieb:
> Author: alexandre.vassalotti
> Date: Mon Apr 14 22:51:05 2008
> New Revision: 62342
> 
> Log:
> Improved bytes_extend() to avoid making a full copy of the temporary
> buffer. This also makes the code slightly cleaner.

Changes to Objects/bytesobject.c should be applied to the trunk and
merged into the py3k branch via svnmerge.py.

We need to agree on a policy how we are going to sync the trunk and py3k
for new code like bytesobject.c and io.py. The former is easy because
the file is almost identical. The later is going to be hard because 2.6
doesn't have annotations.

Collin:
How hard is it to write a fixer that removes all annotations from
functions? A set of small 3to2 fixers for annotations and metaclasses
would make the syncing job much easier.

Christian

From collinw at gmail.com  Mon Apr 14 23:18:55 2008
From: collinw at gmail.com (Collin Winter)
Date: Mon, 14 Apr 2008 14:18:55 -0700
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <4803C735.10100@cheimes.de>
References: <20080414205105.B45C11E4005@bag.python.org>
	<4803C735.10100@cheimes.de>
Message-ID: <43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>

On Mon, Apr 14, 2008 at 2:05 PM, Christian Heimes <lists at cheimes.de> wrote:
> alexandre.vassalotti schrieb:
>
> > Author: alexandre.vassalotti
>  > Date: Mon Apr 14 22:51:05 2008
>  > New Revision: 62342
>  >
>  > Log:
>  > Improved bytes_extend() to avoid making a full copy of the temporary
>  > buffer. This also makes the code slightly cleaner.
>
>  Changes to Objects/bytesobject.c should be applied to the trunk and
>  merged into the py3k branch via svnmerge.py.
>
>  We need to agree on a policy how we are going to sync the trunk and py3k
>  for new code like bytesobject.c and io.py. The former is easy because
>  the file is almost identical. The later is going to be hard because 2.6
>  doesn't have annotations.
>
>  Collin:
>  How hard is it to write a fixer that removes all annotations from
>  functions? A set of small 3to2 fixers for annotations and metaclasses
>  would make the syncing job much easier.

It should be pretty easy. I'm working on a 2to3 metaclass fixer, which
could provide guidance for a 3to2 fixer.

Do we want to take this opportunity to create a real 3to2 project in
the sandbox? If so, I'd like to refactor lib2to3 into its own project,
then import that into 2to3 and 3to2. I can do the work.

Collin

From schmir at gmail.com  Mon Apr 14 23:18:15 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 14 Apr 2008 23:18:15 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <ca471dc20804141122g20aededu66447acc677ed439@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<ca471dc20804141122g20aededu66447acc677ed439@mail.gmail.com>
Message-ID: <932f8baf0804141418q2b22bb97k74c6e12c2e9518b@mail.gmail.com>

On Mon, Apr 14, 2008 at 8:22 PM, Guido van Rossum <guido at python.org> wrote:

> Eek! Please use the bug tracker.
>

I 've made some comments on: http://bugs.python.org/issue1092502 (which is
the original issue). However I cannot reopen this issue.

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080414/36746efb/attachment-0001.htm 

From martin at v.loewis.de  Mon Apr 14 23:26:26 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 14 Apr 2008 23:26:26 +0200
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
References: <20080414205105.B45C11E4005@bag.python.org>	<4803C735.10100@cheimes.de>
	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
Message-ID: <4803CC02.8060400@v.loewis.de>

> Do we want to take this opportunity to create a real 3to2 project in
> the sandbox? If so, I'd like to refactor lib2to3 into its own project,
> then import that into 2to3 and 3to2. I can do the work.

In that case, I would propose that the copy in the Python trunk becomes
official, and the other copies use svn:externals, or merge tracking.

Regards,
Martin

From schmir at gmail.com  Mon Apr 14 23:29:21 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 14 Apr 2008 23:29:21 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
Message-ID: <932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>

On Mon, Apr 14, 2008 at 8:10 PM, Curt Hagenlocher <curt at hagenlocher.org>
wrote:

> On Mon, Apr 14, 2008 at 9:12 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> >
> > I've tracked it down to this change:
> > http://hgpy.de/py/release25-maint/rev/e9446c6ab3cd
> > this is svn revision 61009.
> > [...]
> > self._rbufsize if 1, and so the code reads one byte at a time
>
> The change is correct, but exposes a flaw earlier in the same method.
> "_rbufsize == 1" represents a request to buffer "by line", which is
> clearly irrelevant in this context.  A request to read n bytes should
> just use the default buffer size if buffering "by line".  Sample patch
> is attached.
>

Sorry to reply on the mailing list. But this change is wrong.
e.g. if you're using a buffer size of 16 bytes and try to read 256 bytes, it
should call recv with a value of 256 and not call recv 16 times with a value
of 16.
However, there should be an upper limit (as shown by the imap bug).

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080414/34506a5b/attachment.htm 

From collinw at gmail.com  Mon Apr 14 23:35:16 2008
From: collinw at gmail.com (Collin Winter)
Date: Mon, 14 Apr 2008 14:35:16 -0700
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <4803CC02.8060400@v.loewis.de>
References: <20080414205105.B45C11E4005@bag.python.org>
	<4803C735.10100@cheimes.de>
	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
	<4803CC02.8060400@v.loewis.de>
Message-ID: <43aa6ff70804141435h2d487790u70e4fed0c7cfbdee@mail.gmail.com>

On Mon, Apr 14, 2008 at 2:26 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Do we want to take this opportunity to create a real 3to2 project in
>  > the sandbox? If so, I'd like to refactor lib2to3 into its own project,
>  > then import that into 2to3 and 3to2. I can do the work.
>
>  In that case, I would propose that the copy in the Python trunk becomes
>  official, and the other copies use svn:externals, or merge tracking.

Can do.

From schmir at gmail.com  Mon Apr 14 23:45:21 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 14 Apr 2008 23:45:21 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <932f8baf0804141418q2b22bb97k74c6e12c2e9518b@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<ca471dc20804141122g20aededu66447acc677ed439@mail.gmail.com>
	<932f8baf0804141418q2b22bb97k74c6e12c2e9518b@mail.gmail.com>
Message-ID: <932f8baf0804141445k31ead5baic0201759a49d6714@mail.gmail.com>

On Mon, Apr 14, 2008 at 11:18 PM, Ralf Schmitt <schmir at gmail.com> wrote:

>
> On Mon, Apr 14, 2008 at 8:22 PM, Guido van Rossum <guido at python.org>
> wrote:
>
> > Eek! Please use the bug tracker.
> >
>
> I 've made some comments on: http://bugs.python.org/issue1092502 (which is
> the original issue). However I cannot reopen this issue.
>

Curt opened another bug for it:
http://bugs.python.org/issue2632
someone should change the priority.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080414/423d92dd/attachment.htm 

From lists at cheimes.de  Mon Apr 14 23:56:41 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 14 Apr 2008 23:56:41 +0200
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
References: <20080414205105.B45C11E4005@bag.python.org>	
	<4803C735.10100@cheimes.de>
	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
Message-ID: <4803D319.5010000@cheimes.de>

Collin Winter schrieb:
> It should be pretty easy. I'm working on a 2to3 metaclass fixer, which
> could provide guidance for a 3to2 fixer.
> 
> Do we want to take this opportunity to create a real 3to2 project in
> the sandbox? If so, I'd like to refactor lib2to3 into its own project,
> then import that into 2to3 and 3to2. I can do the work.

Good idea! A 3to2 project is going to make backporting io.py and other
new stuff less painful and much faster. For the io.py backport I spent
most time on removing annotations and replacing "" with u"".

What needs to be done?

* remove funtion annotation

* Add object to all empty class definition

* replace class Egg(metaclass=Spam) with class
Egg(object):\n__metaclass__ = Spam

* Add __future__ imports for print_function and unicode literals

Anything else?

Christian

From curt at hagenlocher.org  Tue Apr 15 00:19:35 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 14 Apr 2008 15:19:35 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
Message-ID: <d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>

On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <schmir at gmail.com> wrote:
>
> Sorry to reply on the mailing list. But this change is wrong.
> e.g. if you're using a buffer size of 16 bytes and try to read 256 bytes, it
> should call recv with a value of 256 and not call recv 16 times with a value
> of 16.
> However, there should be an upper limit (as shown by the imap bug).

There is an upper limit.  It's called "the buffer size".  If someone
specifies a buffer size of 16 bytes, it means "read 16 bytes at a
time".  I don't know why someone would want such a small buffer size,
but presumably they have their reasons.

The only reason "min" is a problem is that there's standard library
code passing a zero to socket.makefile, which gets turned into a
bufsize of 1 by the constructor.  I actually agree with Bill Janssen
-- __init__ is where the real problem lies.  But I think the change to
read() is safer.

--
Curt Hagenlocher
curt at hagenlocher.org

From schmir at gmail.com  Tue Apr 15 00:57:05 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 00:57:05 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
Message-ID: <932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>

On Tue, Apr 15, 2008 at 12:19 AM, Curt Hagenlocher <curt at hagenlocher.org>
wrote:

> On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <schmir at gmail.com> wrote:
> >
> > Sorry to reply on the mailing list. But this change is wrong.
> > e.g. if you're using a buffer size of 16 bytes and try to read 256
> bytes, it
> > should call recv with a value of 256 and not call recv 16 times with a
> value
> > of 16.
> > However, there should be an upper limit (as shown by the imap bug).
>
> There is an upper limit.  It's called "the buffer size".  If someone
> specifies a buffer size of 16 bytes, it means "read 16 bytes at a
> time".  I don't know why someone would want such a small buffer size,
> but presumably they have their reasons.
>

No, I don't agree. To me buffer size means buffer up to buffer_size bytes in
memory.
It does not mean that it should read only buffer_size bytes at once when
asked to read more bytes than buffer size.

 The upper limit I was talking about is the buffer size limit of the
operating system, i.e. the operating system will at a maximum return N bytes
from recv call. It doesn't make sense to ask for more then, and the original
problem with imaplip asking for 10MB of data and then realloc'ing that
buffer would be gone.


> The only reason "min" is a problem is that there's standard library
> code passing a zero to socket.makefile, which gets turned into a
> bufsize of 1 by the constructor.  I actually agree with Bill Janssen
> -- __init__ is where the real problem lies.  But I think the change to
> read() is safer.
>

again no, if I pass in 4 as buffer size, I don't expect the system to make
1024 calls to recv when I want to read 4096 bytes.

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/dba04a91/attachment.htm 

From guido at python.org  Tue Apr 15 01:19:44 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 16:19:44 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
	<932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
Message-ID: <ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>

On Mon, Apr 14, 2008 at 3:57 PM, Ralf Schmitt <schmir at gmail.com> wrote:
>
>
>
> On Tue, Apr 15, 2008 at 12:19 AM, Curt Hagenlocher <curt at hagenlocher.org>
> wrote:
> >
> > On Mon, Apr 14, 2008 at 2:29 PM, Ralf Schmitt <schmir at gmail.com> wrote:
> > >
> > > Sorry to reply on the mailing list. But this change is wrong.
> > > e.g. if you're using a buffer size of 16 bytes and try to read 256
> bytes, it
> > > should call recv with a value of 256 and not call recv 16 times with a
> value
> > > of 16.
> > > However, there should be an upper limit (as shown by the imap bug).
> >
> > There is an upper limit.  It's called "the buffer size".  If someone
> > specifies a buffer size of 16 bytes, it means "read 16 bytes at a
> > time".  I don't know why someone would want such a small buffer size,
> > but presumably they have their reasons.
> >
>
> No, I don't agree. To me buffer size means buffer up to buffer_size bytes in
> memory.
> It does not mean that it should read only buffer_size bytes at once when
> asked to read more bytes than buffer size.
>
>  The upper limit I was talking about is the buffer size limit of the
> operating system, i.e. the operating system will at a maximum return N bytes
> from recv call. It doesn't make sense to ask for more then, and the original
> problem with imaplip asking for 10MB of data and then realloc'ing that
> buffer would be gone.
>
>
> >
> > The only reason "min" is a problem is that there's standard library
> > code passing a zero to socket.makefile, which gets turned into a
> > bufsize of 1 by the constructor.  I actually agree with Bill Janssen
> > -- __init__ is where the real problem lies.  But I think the change to
> > read() is safer.
> >
>
> again no, if I pass in 4 as buffer size, I don't expect the system to make
> 1024 calls to recv when I want to read 4096 bytes.

But why was imaplib apparently specifying 10MB? Did it know there was
that much data? Or did it just not want to bother looping over all the
data in smaller buffer increments (e.g. 64K, which is probably the max
of what most TCP stacks will give you)?

If I'm right with my hunch that the TCP stack will probably clamp at
64K, perhaps we should use min(system limit, max(requested size,
buffer size))?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From schmir at gmail.com  Tue Apr 15 01:28:06 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 01:28:06 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
	<932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
	<ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
Message-ID: <932f8baf0804141628k19993582jeabca7d31806f2e1@mail.gmail.com>

On Tue, Apr 15, 2008 at 1:19 AM, Guido van Rossum <guido at python.org> wrote:

>
> But why was imaplib apparently specifying 10MB? Did it know there was
> that much data? Or did it just not want to bother looping over all the
> data in smaller buffer increments (e.g. 64K, which is probably the max
> of what most TCP stacks will give you)?
>

Well, calling read with a size of 10MB should be possible. The problem is
that this value ended up
inside the recv call, which then did the malloc/realloc calls.


>
> If I'm right with my hunch that the TCP stack will probably clamp at
> 64K, perhaps we should use min(system limit, max(requested size,
> buffer size))?
>

yes, this is what I was trying to explain.


>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/1271d926/attachment.htm 

From curt at hagenlocher.org  Tue Apr 15 01:41:27 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 14 Apr 2008 16:41:27 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
	<932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
	<ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
Message-ID: <d2155e360804141641n543ebc8bp77a16c1289b92e45@mail.gmail.com>

On Mon, Apr 14, 2008 at 4:19 PM, Guido van Rossum <guido at python.org> wrote:
>
> But why was imaplib apparently specifying 10MB? Did it know there was
> that much data? Or did it just not want to bother looping over all the
> data in smaller buffer increments (e.g. 64K, which is probably the max
> of what most TCP stacks will give you)?

I'm going to guess that the code in question is

                size = int(self.mo.group('size'))
                if __debug__:
                    if self.debug >= 4:
                        self._mesg('read literal size %s' % size)
                data = self.read(size)

It's reading however many bytes are reported by the server as the size.

> If I'm right with my hunch that the TCP stack will probably clamp at
> 64K, perhaps we should use min(system limit, max(requested size,
> buffer size))?

I have indeed missed the point of the read buffer size.  This would work.

--
Curt Hagenlocher
curt at hagenlocher.org

From greg.ewing at canterbury.ac.nz  Tue Apr 15 03:59:36 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 15 Apr 2008 13:59:36 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
Message-ID: <48040C08.504@canterbury.ac.nz>

Guido van Rossum wrote:
> I don't think of EOFError as an environmental error... This is quite
 > a different level of error than what EnvironmentError typically means

I think it depends. Any "expected" EOFErrors are going to be
caught by the surrounding code before propagating very far.
An *uncaught* EOFError probably means that a file was shorter
than you expected it to be, which counts as an environmental
error to my way of thinking.

My current coding style involves wrapping an "except EnvironmentError"
around any major operation and reporting it as a "File could not be
read/written/whatever because..." kind of message. Having
EOFError get missed by that would be a nuisance.

-- 
Greg

From guido at python.org  Tue Apr 15 04:11:26 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Apr 2008 19:11:26 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
In-Reply-To: <48040C08.504@canterbury.ac.nz>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
Message-ID: <ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>

On Mon, Apr 14, 2008 at 6:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>  > I don't think of EOFError as an environmental error... This is quite
>
>  > a different level of error than what EnvironmentError typically means
>
>  I think it depends. Any "expected" EOFErrors are going to be
>  caught by the surrounding code before propagating very far.
>  An *uncaught* EOFError probably means that a file was shorter
>  than you expected it to be, which counts as an environmental
>  error to my way of thinking.

No, that's some kind of parsing error. EnvironmentError doesn't
concern itself with the contents of files.

>  My current coding style involves wrapping an "except EnvironmentError"
>  around any major operation and reporting it as a "File could not be
>  read/written/whatever because..." kind of message. Having
>  EOFError get missed by that would be a nuisance.

But what operations raise EOFError? Surely you're not using
raw_input()? It's really only there for teaching.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Tue Apr 15 04:21:40 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 14 Apr 2008 21:21:40 -0500
Subject: [Python-Dev] NeedsReview keyword
Message-ID: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>

I think it would be useful for the tracker to grow a "NeedsReview"
keyword. I realize the "patch" keyword does some of this, but it may
just represent some initial or trivial work. "NeedsReview" should
represent a mature patch that some senior dev needs to look hard at
and make the choice.

-- 
Cheers,
Benjamin Peterson

From nnorwitz at gmail.com  Tue Apr 15 06:19:02 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 14 Apr 2008 21:19:02 -0700
Subject: [Python-Dev] Python 2.4.4/2.4.5 test_pty failure on Solaris 10
In-Reply-To: <18432.63770.323783.565994@montanaro-dyndns-org.local>
References: <18432.63770.323783.565994@montanaro-dyndns-org.local>
Message-ID: <ee2a432c0804142119x4bb6ed4se62e2b146e0b3c24@mail.gmail.com>

On Sat, Apr 12, 2008 at 11:02 AM,  <skip at pobox.com> wrote:
>
>  I know this is old stuff, but...
>
>  I want to update our Python 2.4 installation at work from 2.4.2 to 2.4.5
>  (the latest 2.4 source release).  I get a test failure for test_pty, an
>  extra ^M at the end of one line.  I don't get a failure in the 2.4.2
>  installation, but the 2.4.4 and 2.4.5 both fail this test.  Looking at the
>  code in test_pty.py, it appears to me that r43570 fixed things for OSF/1 and
>  IRIX which both do weird things with output while breaking things for any
>  other platform by suppressing the \r\n -> \n mapping which used to be
>  performed for all platforms.  So, for Solaris, that mapping doesn't happen
>  and the actual and expected outputs don't agree.

This was probably me.  Perhaps a fix wasn't backported?  I notice the
2.5 version of the test changed from the 2.4 version and does a
str.replace rather than changing the last chars of the string.  You
can try using the 2.5 version and my guess is it will work (ie, the
test will pass).  The change is in normalize_output.

n

From nnorwitz at gmail.com  Tue Apr 15 07:06:51 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 14 Apr 2008 22:06:51 -0700
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <4803D319.5010000@cheimes.de>
References: <20080414205105.B45C11E4005@bag.python.org>
	<4803C735.10100@cheimes.de>
	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
	<4803D319.5010000@cheimes.de>
Message-ID: <ee2a432c0804142206j65978c2dse4a9645d9dd2d351@mail.gmail.com>

On Mon, Apr 14, 2008 at 2:56 PM, Christian Heimes <lists at cheimes.de> wrote:
>
>  Good idea! A 3to2 project is going to make backporting io.py and other
>  new stuff less painful and much faster. For the io.py backport I spent
>  most time on removing annotations and replacing "" with u"".
>
>  What needs to be done?
>
>  * remove funtion annotation
>
>  * Add object to all empty class definition
>
>  * replace class Egg(metaclass=Spam) with class
>  Egg(object):\n__metaclass__ = Spam
>
>  * Add __future__ imports for print_function and unicode literals
>
>  Anything else?

Iteration with the dict methods (e.g., keys -> iterkeys()),
map/zip/filter returning iterator rather than list.
int -> (int, long)
str -> basestring or (str, unicode)
__bool__ -> __nonzero__
exec/execfile
input -> rawinput

Most things that have a fixer in 2to3 would also require one in 3to2.
Only things that work in backwards compatible ways like apply/has_key
removal, etc don't need a 3to2 fixer.  Although most of these 3to2
fixers are probably pretty low priority as they are not real likely to
be used in the python code base.  They are still needed for general
user code.

n

From martin at v.loewis.de  Tue Apr 15 07:21:07 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Apr 2008 07:21:07 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>
Message-ID: <48043B43.7070303@v.loewis.de>

> I think it would be useful for the tracker to grow a "NeedsReview"
> keyword. I realize the "patch" keyword does some of this, but it may
> just represent some initial or trivial work. "NeedsReview" should
> represent a mature patch that some senior dev needs to look hard at
> and make the choice.

Not sure what problem that would solve. Over time, I would expect that
any open patch also grows the "NeedsReview" keyword, making the keyword
pointless. If somebody specifically should review a certain proposed
change, the change should be assigned to that person. If someone in
a group should review, they should be contacted by email.

Regards,
Martin

From schmir at gmail.com  Tue Apr 15 07:31:07 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 07:31:07 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <48043B43.7070303@v.loewis.de>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>
	<48043B43.7070303@v.loewis.de>
Message-ID: <932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>

On Tue, Apr 15, 2008 at 7:21 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > I think it would be useful for the tracker to grow a "NeedsReview"
> > keyword. I realize the "patch" keyword does some of this, but it may
> > just represent some initial or trivial work. "NeedsReview" should
> > represent a mature patch that some senior dev needs to look hard at
> > and make the choice.
>
> Not sure what problem that would solve. Over time, I would expect that
> any open patch also grows the "NeedsReview" keyword, making the keyword
> pointless. If somebody specifically should review a certain proposed
> change, the change should be assigned to that person. If someone in
> a group should review, they should be contacted by email.
>

I think it would be nice if that patch keyword could be set by non-admins.
This would mean I didn't have to write to the mailing list asking for people
to look at
some specific bug. Like "did someone look at
http://bugs.python.org/issue2122. This isssue is about mmap.flush not
raising an exception on errors. which I think is a rather severe". (btw. can
someone please look at it? :) )

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/36aeb655/attachment-0001.htm 

From nnorwitz at gmail.com  Tue Apr 15 07:43:06 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 14 Apr 2008 22:43:06 -0700
Subject: [Python-Dev] Next monthly sprint/bugfix day?
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22BEB730@EXMBX04.exchhosting.com>
Message-ID: <ee2a432c0804142243p38cd1123geb76a814613a41a6@mail.gmail.com>

On Wed, Apr 9, 2008 at 7:12 AM, Trent Nelson <tnelson at onresolve.com> wrote:
> Hi,
>
>  Is there another online sprint/bugfix day in the pipeline?  If not, can there be? ;-)

Trent, I think you just volunteered to lead it. :-)

We should either do it this weekend Apr 19-20 or wait until after the
release.  The first available date should be May 10.

The schedule http://www.python.org/dev/peps/pep-0361/ has the upcoming
May 7 release as the last alpha.  That means we are getting closer to
an API freeze.  Anything that might change an API for 2.6/3.0 should
be addressed sooner rather than later.  If we have to change an API
before release, please update the bug tracker priority to "release
blocker".

n

From martin at v.loewis.de  Tue Apr 15 07:54:48 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Apr 2008 07:54:48 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>	<48043B43.7070303@v.loewis.de>
	<932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>
Message-ID: <48044328.6040207@v.loewis.de>

> I think it would be nice if that patch keyword could be set by non-admins.
> This would mean I didn't have to write to the mailing list asking for
> people to look at
> some specific bug. Like "did someone look at
> http://bugs.python.org/issue2122.

Just name your patch files .patch or .diff the next time, not .txt, and
the keyword will get automatically set.

 This isssue is about mmap.flush not
> raising an exception on errors. which I think is a rather severe". (btw.
> can someone please look at it? :) )

I've added the patch keyword. I don't think the bug is rather severe,
as it only affects the mmap module. I also don't see how this could
cause data loss if the application works correctly.

Regards,
Martin

From schmir at gmail.com  Tue Apr 15 08:19:01 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 08:19:01 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <48044328.6040207@v.loewis.de>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>
	<48043B43.7070303@v.loewis.de>
	<932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>
	<48044328.6040207@v.loewis.de>
Message-ID: <932f8baf0804142319p19ec1565ye364ad98b7674a3e@mail.gmail.com>

On Tue, Apr 15, 2008 at 7:54 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

>
> Just name your patch files .patch or .diff the next time, not .txt, and
> the keyword will get automatically set.
>

fine. I used .txt cause I wanted to view it in my browser (without the
browser asking me for an application)


>
>  This isssue is about mmap.flush not
> > raising an exception on errors. which I think is a rather severe". (btw.
> > can someone please look at it? :) )
>
> I've added the patch keyword. I don't think the bug is rather severe,
>

thanks.


> as it only affects the mmap module. I also don't see how this could
> cause data loss if the application works correctly.
>

the flush fails but the programs fails to recognize it? i.e. the program
assumes the data is written to disk but it isn't?

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/f40f4f7e/attachment.htm 

From p.f.moore at gmail.com  Tue Apr 15 10:57:52 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Apr 2008 09:57:52 +0100
Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources
	module)
In-Reply-To: <ca471dc20803191206j2eae1be3rc5745f955d8e0dce@mail.gmail.com>
References: <ca471dc20803161613h75cdec13t65dc62e87bfe271c@mail.gmail.com>
	<20080317151637.532D23A409D@sparrow.telecommunity.com>
	<ca471dc20803170853s490339f2g20bb1d95122ff77f@mail.gmail.com>
	<20080317161305.22B183A409D@sparrow.telecommunity.com>
	<ca471dc20803170931o6cb7581fj674451c8dd653ee8@mail.gmail.com>
	<47DEC0EB.3000404@palladion.com>
	<440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com>
	<ca471dc20803191025i7ea5af15wc0bdcc4a47632720@mail.gmail.com>
	<79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com>
	<ca471dc20803191206j2eae1be3rc5745f955d8e0dce@mail.gmail.com>
Message-ID: <79990c6b0804150157n558c7572x17f33582e6a585a9@mail.gmail.com>

On 19/03/2008, Guido van Rossum <guido at python.org> wrote:
> >  I'm currently working on an addition to pkgutil to provide this type
> >  of function. I'm considering going a little further (adding functions
> >  to get a file-like object, test for existence, and list available
> >  resources, modelled on the pkg_resources functions - but these extra
> >  ones are not supported by the loader protocol, so I'm undecided as to
> >  whether it's worth it, I'll see how complex the code gets).
>
> I'd only do what __loader__ offers. People can always wrap a StringIO around it.
>
> >  Once I have a patch, I'll post it to the tracker. What's the best
> >  approach? Code a patch for 3.0 and backport, or code for 2.6 and let
> >  the merging process do its stuff?
>
> Code for 2.6, let the merge do its thing.

I've had a patch in http://bugs.python.org/issue2439 for a few weeks
now. It just adds a get_data function, on the basis that that's all
the loader API offers. I think it's complete - Phillip Eby helped
thrash out a couple of problems.

Is there anything I can do to get it applied, or should I just leave
it now for someone to decide if they care enough? (As it's a library
change, I don't know to what extent the "no API changes after the next
alpha" rule will apply).

Paul.

From stefan_ml at behnel.de  Tue Apr 15 11:21:28 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 15 Apr 2008 11:21:28 +0200
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <ee2a432c0804142206j65978c2dse4a9645d9dd2d351@mail.gmail.com>
References: <20080414205105.B45C11E4005@bag.python.org>	<4803C735.10100@cheimes.de>	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>	<4803D319.5010000@cheimes.de>
	<ee2a432c0804142206j65978c2dse4a9645d9dd2d351@mail.gmail.com>
Message-ID: <fu1s2n$i84$1@ger.gmane.org>

Neal Norwitz wrote:
> Iteration with the dict methods (e.g., keys -> iterkeys()),
> map/zip/filter returning iterator rather than list.

That's only an optimisation, it's not functionally required. A list behaves
like an iterator in most use cases, so it's rather unlikely that Py3 code will
break in the backport because of this (and it's very unlikely that static
analysis can catch the cases where it breaks). There should be a rule to
optimise "list(map(...))" into "map(...)" and "list(x.keys())" into plain
"x.keys()" etc., but I don't think there is much more to do here.


> int -> (int, long)

Is there any case where this might be required? I don't see any reason why
back-converted Py3 code should break here. What would "long()" be needed for
in working Py3 code that "int()" doesn't provide in Py2?

Although you might have been referring to "isinstance(x, int)" in Py3?


> str -> basestring or (str, unicode)

This is an issue, although I think that Py3 is explicit enough here to make
this easy to handle by static renaming (and maybe optimising "isinstance(s,
(str, bytes))" into "..., basestring))").


> __bool__ -> __nonzero__
> exec/execfile
> input -> rawinput

Also valid issues that can be handled through renaming and static syntactic
adjustments.


> Most things that have a fixer in 2to3 would also require one in 3to2.

I think the more explicit syntax in Py3 will actually make it easier to
back-convert the code statically from 3to2 than to migrate it from 2to3.

Stefan


From ncoghlan at gmail.com  Tue Apr 15 11:57:27 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Apr 2008 19:57:27 +1000
Subject: [Python-Dev] [Distutils] PEP 365 (Adding the
	pkg_resources	module)
In-Reply-To: <79990c6b0804150157n558c7572x17f33582e6a585a9@mail.gmail.com>
References: <ca471dc20803161613h75cdec13t65dc62e87bfe271c@mail.gmail.com>	<20080317151637.532D23A409D@sparrow.telecommunity.com>	<ca471dc20803170853s490339f2g20bb1d95122ff77f@mail.gmail.com>	<20080317161305.22B183A409D@sparrow.telecommunity.com>	<ca471dc20803170931o6cb7581fj674451c8dd653ee8@mail.gmail.com>	<47DEC0EB.3000404@palladion.com>	<440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com>	<ca471dc20803191025i7ea5af15wc0bdcc4a47632720@mail.gmail.com>	<79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com>	<ca471dc20803191206j2eae1be3rc5745f955d8e0dce@mail.gmail.com>
	<79990c6b0804150157n558c7572x17f33582e6a585a9@mail.gmail.com>
Message-ID: <48047C07.1020807@gmail.com>

Paul Moore wrote:
> Is there anything I can do to get it applied, or should I just leave
> it now for someone to decide if they care enough? (As it's a library
> change, I don't know to what extent the "no API changes after the next
> alpha" rule will apply).

I'm looking into it now - assuming it applies cleanly and the tests run 
happily afterwards (and I don't see any reason it won't after reading 
the patch), it should be checked in later tonight.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From sergiodj at linux.vnet.ibm.com  Tue Apr 15 19:48:07 2008
From: sergiodj at linux.vnet.ibm.com (=?ISO-8859-1?Q?S=E9rgio?= Durigan =?ISO-8859-1?Q?J=FAnior?=)
Date: Tue, 15 Apr 2008 14:48:07 -0300
Subject: [Python-Dev] weird configure (autotools) setup
In-Reply-To: <20080412100027.GN51167@nexus.in-nomine.org>
References: <20080412100027.GN51167@nexus.in-nomine.org>
Message-ID: <1208281687.5817.15.camel@miki>

Hi Jeroen

On Sat, 2008-04-12 at 12:00 +0200, Jeroen Ruigrok van der Werven wrote:
> Why is CFLAGS in Makefile.pre.in specified as
> CFLAGS=         $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
> whereas that will negate any CFLAGS you pass to configure?

I suggest you read (and follow) this issue:

http://bugs.python.org/issue1628484

Bob and I are discussing the same thing, so if you want to help us in
this effort of making Python's build system more flexible and correct,
you're really welcome.

Regards,

-- 
S?rgio Durigan J?nior
Linux on Power Toolchain - Software Engineer
Linux Technology Center - LTC
IBM Brazil


From tnelson at onresolve.com  Tue Apr 15 22:13:35 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 15 Apr 2008 13:13:35 -0700
Subject: [Python-Dev] Deco
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22E7536B@EXMBX04.exchhosting.com>

Neal, Martin,

We're in the process of decommissioning the box the 'x86 FreeBSD 3' build slave is running on, can you remove it from the list?  Our new FreeBSD 7.0 server is up, which we're slowly migrating to, and I'll be able to set a slave up on that probably some time next week once we've moved our production stuff over.

        Trent.

From martin at v.loewis.de  Tue Apr 15 22:45:58 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Apr 2008 22:45:58 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <932f8baf0804142319p19ec1565ye364ad98b7674a3e@mail.gmail.com>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>	
	<48043B43.7070303@v.loewis.de>	
	<932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>	
	<48044328.6040207@v.loewis.de>
	<932f8baf0804142319p19ec1565ye364ad98b7674a3e@mail.gmail.com>
Message-ID: <48051406.80501@v.loewis.de>

>     Just name your patch files .patch or .diff the next time, not .txt, and
>     the keyword will get automatically set.
> 
> 
> fine. I used .txt cause I wanted to view it in my browser (without the
> browser asking me for an application)

.diff would have done the same thing.

>     as it only affects the mmap module. I also don't see how this could
>     cause data loss if the application works correctly.
> 
> 
> the flush fails but the programs fails to recognize it? i.e. the program
> assumes the data is written to disk but it isn't?

Why would the program fail to recognize it? It should just look at the
result being returned.

I agree that this is fairly unpythonic, but I also think that there is
any data loss that this may cause is the application's fault.

Regards,
Martin

From schmir at gmail.com  Tue Apr 15 23:03:57 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 23:03:57 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <48051406.80501@v.loewis.de>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>
	<48043B43.7070303@v.loewis.de>
	<932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>
	<48044328.6040207@v.loewis.de>
	<932f8baf0804142319p19ec1565ye364ad98b7674a3e@mail.gmail.com>
	<48051406.80501@v.loewis.de>
Message-ID: <932f8baf0804151403m1b35e1dfvf49e0b218c5862a9@mail.gmail.com>

> >
> > the flush fails but the programs fails to recognize it? i.e. the program
> > assumes the data is written to disk but it isn't?
>
> Why would the program fail to recognize it? It should just look at the
> result being returned.
>

sorry no. everything else raises an error. this is not documented (as far as
I can see) and mmap.flush returns 0 on unix if it succeeds, and raises an
exception on unix if it fails.

on windows it returns 1 if it succeeds and 0 if it doesn't (One more time:
on unix it returns 0 if it succeeds).

Regards,
- Ralf


> I agree that this is fairly unpythonic, but I also think that there is
> any data loss that this may cause is the application's fault.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/50940dc1/attachment.htm 

From martin at v.loewis.de  Tue Apr 15 23:20:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Apr 2008 23:20:20 +0200
Subject: [Python-Dev] NeedsReview keyword
In-Reply-To: <932f8baf0804151403m1b35e1dfvf49e0b218c5862a9@mail.gmail.com>
References: <1afaf6160804141921h239689c6s7f37e6af3ed70ab2@mail.gmail.com>	
	<48043B43.7070303@v.loewis.de>	
	<932f8baf0804142231j611e7803j12103388d4c163c5@mail.gmail.com>	
	<48044328.6040207@v.loewis.de>	
	<932f8baf0804142319p19ec1565ye364ad98b7674a3e@mail.gmail.com>	
	<48051406.80501@v.loewis.de>
	<932f8baf0804151403m1b35e1dfvf49e0b218c5862a9@mail.gmail.com>
Message-ID: <48051C14.6030002@v.loewis.de>

>     > the flush fails but the programs fails to recognize it? i.e. the
>     program
>     > assumes the data is written to disk but it isn't?
> 
>     Why would the program fail to recognize it? It should just look at the
>     result being returned.
> 
> 
> sorry no.

Sorry no what?

Regards,
Martin

From schmir at gmail.com  Tue Apr 15 23:37:32 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 15 Apr 2008 23:37:32 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
Message-ID: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>

On Tue, Apr 15, 2008 at 11:20 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> >     > the flush fails but the programs fails to recognize it? i.e. the
> >     program
> >     > assumes the data is written to disk but it isn't?
> >
> >     Why would the program fail to recognize it? It should just look at
> the
> >     result being returned.
> >
> >
> > sorry no.
>
> Sorry no what?
>

mmap.flush returns different values for windows/unix like platforms in case
it succeeds.
mmap.flush raises an exception on unix like platforms for errors.
mmap.flush returns 0 on windows for errors. This is the value which is
returned on unix like platforms for a successful call.
The documentation for mmap.flush does not even mention a return value.

so, still: sorry no, the application should not just look at the result
being returned. The mmap.flush method should be fixed.



> Regards,
> Martin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080415/66539465/attachment.htm 

From martin at v.loewis.de  Tue Apr 15 23:49:17 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Apr 2008 23:49:17 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
Message-ID: <480522DD.2020800@v.loewis.de>

> so, still: sorry no, the application should not just look at the result
> being returned.

If it doesn't want to lose data, it *has* to, because of the way it's
currently implemented. There is no other way (other than ignoring the
error) in Python 2.5.x and earlier.

> The mmap.flush method should be fixed.

I never debated that. I debated whether that is a "rather severe" issue,
which I still don't think it is.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Tue Apr 15 23:53:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Apr 2008 09:53:22 +1200
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
Message-ID: <480523D2.2070705@canterbury.ac.nz>

Brad Miller wrote:
> It was suggested that a broader discussion might be in order around the 
> issue of iterators and how they are displayed in the command line 
> interpreter.

Whatever is done, I don't think it would be a good idea
to make the str() of these things look *too* much like
a list. I think that would make things more confusing
for a newcomer rather than less.

The way they are, at least it's obvious that they're
something special.

-- 
Greg

From guido at python.org  Wed Apr 16 00:12:52 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2008 15:12:52 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <480523D2.2070705@canterbury.ac.nz>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
Message-ID: <ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>

That's why I proposed <0, 1, ..., 9> for repr(range(10)). (And I meant
the '...' literally, i.e. if there are more than 4 values, replace all
but the first two and the last with three dots. And yes, I mean that
str(range(4)) == '<0, 1, 2, 3>' but str(range(5)) == '<0, 1, ..., 4>'.

I'm not at all sure that we should go the same way for dict views
though. They are quite different beasts -- the fact that they change
depending on the underlying dict ought to be somehow emphasized, and
I'd be happier to keep these as they are in 3.0a4.

--Guido

On Tue, Apr 15, 2008 at 2:53 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Brad Miller wrote:
>  > It was suggested that a broader discussion might be in order around the
>  > issue of iterators and how they are displayed in the command line
>  > interpreter.
>
>  Whatever is done, I don't think it would be a good idea
>  to make the str() of these things look *too* much like
>  a list. I think that would make things more confusing
>  for a newcomer rather than less.
>
>  The way they are, at least it's obvious that they're
>  something special.
>
>  --
>  Greg
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Apr 16 00:27:13 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Apr 2008 10:27:13 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
Message-ID: <48052BC1.40202@canterbury.ac.nz>

Guido van Rossum wrote:

> No, that's some kind of parsing error. EnvironmentError doesn't
> concern itself with the contents of files.

Often I raise EnvironmentErrors of my own to signal
parsing errors. This makes it easy to wrap everything
in a try-except that catches anything that's the user's
fault rather than the program's.

> But what operations raise EOFError? Surely you're not using
> raw_input()? It's really only there for teaching.

I'm fairly sure there are some others, although I
can't point to them on the spur of the moment.

However, thinking about it a bit more, anything that
calls something that can raise EOFError should be
catching it anyway, so an escaped EOFError represents
a program bug. So it's probably okay.

-- 
Greg

From guido at python.org  Wed Apr 16 00:36:39 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2008 15:36:39 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
In-Reply-To: <48052BC1.40202@canterbury.ac.nz>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
Message-ID: <ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>

On Tue, Apr 15, 2008 at 3:27 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>  > No, that's some kind of parsing error. EnvironmentError doesn't
>  > concern itself with the contents of files.
>
>  Often I raise EnvironmentErrors of my own to signal
>  parsing errors. This makes it easy to wrap everything
>  in a try-except that catches anything that's the user's
>  fault rather than the program's.

Well, that's your problem. That's not what EnvironmentErrors are for.

>  > But what operations raise EOFError? Surely you're not using
>  > raw_input()? It's really only there for teaching.
>
>  I'm fairly sure there are some others, although I
>  can't point to them on the spur of the moment.
>
>  However, thinking about it a bit more, anything that
>  calls something that can raise EOFError should be
>  catching it anyway, so an escaped EOFError represents
>  a program bug. So it's probably okay.

Right.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at pearwood.info  Wed Apr 16 01:40:39 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 16 Apr 2008 09:40:39 +1000
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
Message-ID: <200804160940.40083.steve@pearwood.info>

Hi folks,

Been lurking for a while, this is my first post.

As I recall, this discussion was started because of concern that Python 
3 had introduced features that made it less friendly to beginners. If I 
can quote Brad Miller:

"Several new iterators have appeared in Python 3.0 that makes the  
language less transparent to beginning programmers.  The examples that  
immediately come to mind are shown below..."

It hasn't been that many years since I was a beginner myself, and I'm 
pretty sure that had I seen Guido's suggestion:

> That's why I proposed <0, 1, ..., 9> for repr(range(10)). (And I
> meant the '...' literally, i.e. if there are more than 4 values,
> replace all but the first two and the last with three dots. And yes,
> I mean that str(range(4)) == '<0, 1, 2, 3>' but str(range(5)) == '<0,
> 1, ..., 4>'.

in the interpreter back then, I would have tried writing the literal <0, 
1, 2, 3> as an alternative to [0, 1, 2, 3] or (0, 1, 2, 3) and been 
rather distressed that it didn't do what I expected. As a beginner I 
was rather hazy on the difference between lists and tuples, and I would 
have imagined that <> was just another sort of delimiter pair for a 
list-like thing.

I'd like to say that I'd only make the mistake once, but thinking about 
how often I still write dict(key) instead of dict[key], it's quite 
possible that some people might take a long time to drive the <> 
pseudo-delimiters out of their head.

I'm -1 on <>. In my experience, real beginners aren't going to using 
range() except in the context of for loops, hence are unlikely to be 
printing the range object directly except by accident.

Personally, I find it a feature that this happens in Python 2.x:

>>> xrange(5)
xrange(5)

and would say the same thing about range() in Python 3, regardless of 
whether you wrap it in a str() or a repr().

As for the use of dots to keep stringified-sequences short, I'd vote +1 
except I think that four items is rather short and would prefer the 
dots to come in around six or eight items. That's not entirely an 
aesthetic preference, I also have a reason for thinking that it might 
actually make a difference, but to avoid turning this discussion into 
bike-shedding I'll keep it to myself unless asked.


-- 
Steven

From greg.ewing at canterbury.ac.nz  Wed Apr 16 05:34:44 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Apr 2008 15:34:44 +1200
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
Message-ID: <480573D4.5040708@canterbury.ac.nz>

Guido van Rossum wrote:
> That's why I proposed <0, 1, ..., 9> for repr(range(10)).

My worry is that this will lead a newcomer into thinking
this is some kind of valid expression syntax.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Wed Apr 16 05:40:39 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Apr 2008 15:40:39 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
Message-ID: <48057537.3060003@canterbury.ac.nz>

Guido van Rossum wrote:
>>  Often I raise EnvironmentErrors of my own to signal
>>  parsing errors. 
> 
> Well, that's your problem. That's not what EnvironmentErrors are for.

But it seems like it's what it *should* be for.

I'd be happy to use some other standard exception
type if one existed at the right point in the hierarchy
to catch what I want to catch, but there isn't one.

-- 
Greg

From guido at python.org  Wed Apr 16 06:39:02 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2008 21:39:02 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
In-Reply-To: <48057537.3060003@canterbury.ac.nz>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
Message-ID: <ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>

On Tue, Apr 15, 2008 at 8:40 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>  >>  Often I raise EnvironmentErrors of my own to signal
>  >>  parsing errors.
>  >
>
> > Well, that's your problem. That's not what EnvironmentErrors are for.
>
>  But it seems like it's what it *should* be for.

You are unique in demanding this.

>  I'd be happy to use some other standard exception
>  type if one existed at the right point in the hierarchy
>  to catch what I want to catch, but there isn't one.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Apr 16 06:45:42 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Apr 2008 21:45:42 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <480573D4.5040708@canterbury.ac.nz>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
Message-ID: <ca471dc20804152145q6b5113e4v4263c39616c6a7f2@mail.gmail.com>

On Tue, Apr 15, 2008 at 8:34 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>  > That's why I proposed <0, 1, ..., 9> for repr(range(10)).
>
>  My worry is that this will lead a newcomer into thinking
>  this is some kind of valid expression syntax.

You and Steven D'Aprano both.

So if this is indeed a bad idea, I'm stumped -- the 3.0 status quo is
the best we can do, and teachers will just have to teach their
students to write list(range(10)) if they want to see what it means.
Or they can write an explicit for-loop (always useful on the first
day):

  for i in range(10):
    print i

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From schmir at gmail.com  Wed Apr 16 08:06:49 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Wed, 16 Apr 2008 08:06:49 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <480522DD.2020800@v.loewis.de>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
	<480522DD.2020800@v.loewis.de>
Message-ID: <932f8baf0804152306v184201acgf9362c73086825ec@mail.gmail.com>

On Tue, Apr 15, 2008 at 11:49 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > so, still: sorry no, the application should not just look at the result
> > being returned.
>
> If it doesn't want to lose data, it *has* to, because of the way it's
> currently implemented. There is no other way (other than ignoring the
> error) in Python 2.5.x and earlier.
>

My point is it will ignore this error, because the programmer didn't even
know about the return value.
So it will possibly use data. But, it's all in the bug report and I'm only
repeating myself.


>
> > The mmap.flush method should be fixed.
>
> I never debated that. I debated whether that is a "rather severe" issue,
> which I still don't think it is.
>

I didn't recognize that we discussed this phrase. Sorry if I wasted your
time.

Regards,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080416/231cafe9/attachment.htm 

From arigo at tunes.org  Wed Apr 16 12:15:44 2008
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 16 Apr 2008 12:15:44 +0200
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <480573D4.5040708@canterbury.ac.nz>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
Message-ID: <20080416101544.GA10756@code0.codespeak.net>

Hi Greg,

On Wed, Apr 16, 2008 at 03:34:44PM +1200, Greg Ewing wrote:
> > That's why I proposed <0, 1, ..., 9> for repr(range(10)).
> 
> My worry is that this will lead a newcomer into thinking
> this is some kind of valid expression syntax.

What about the less confusing and more readily generalizable:

   <range object 0, 1, ..., 9>

It would also be helpful IMHO to use this kind of repr for most built-in
iterators and iterables, instead of their mosty-useless default repr.


A bientot,

Armin.

From p.f.moore at gmail.com  Wed Apr 16 14:59:40 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 16 Apr 2008 13:59:40 +0100
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <20080416101544.GA10756@code0.codespeak.net>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
Message-ID: <79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>

On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
> What about the less confusing and more readily generalizable:
>
>   <range object 0, 1, ..., 9>
>
> It would also be helpful IMHO to use this kind of repr for most built-in
> iterators and iterables, instead of their mosty-useless default repr.

I quite like this. But as a non-beginner, I'll freely admit that my
intuitions are suspect :-)

Paul.

From asmodai at in-nomine.org  Wed Apr 16 14:58:45 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 16 Apr 2008 14:58:45 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
Message-ID: <20080416125845.GG91329@nexus.in-nomine.org>

-On [20080415 23:37], Ralf Schmitt (schmir at gmail.com) wrote:
>The documentation for mmap.flush does not even mention a return value.

Fixed in revision 62359.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
With a nuclear fire of Love in our Hearts, rest easy baby, rest easy...

From schmir at gmail.com  Wed Apr 16 15:19:58 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Wed, 16 Apr 2008 15:19:58 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <20080416125845.GG91329@nexus.in-nomine.org>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
	<20080416125845.GG91329@nexus.in-nomine.org>
Message-ID: <932f8baf0804160619q3446203ax40ee303145782c8f@mail.gmail.com>

On Wed, Apr 16, 2008 at 2:58 PM, Jeroen Ruigrok van der Werven <
asmodai at in-nomine.org> wrote:

> -On [20080415 23:37], Ralf Schmitt (schmir at gmail.com) wrote:
> >The documentation for mmap.flush does not even mention a return value.
>
> Fixed in revision 62359.
>

I think documenting this insanity is just insane and doesn't help much.
Anyway, you can now close http://bugs.python.org/issue2122 as wontfix.

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080416/3642075e/attachment.htm 

From asmodai at in-nomine.org  Wed Apr 16 15:11:13 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 16 Apr 2008 15:11:13 +0200
Subject: [Python-Dev] Unreachable code in mmapmodule.c?
Message-ID: <20080416131113.GH91329@nexus.in-nomine.org>

Reading through Modules/mmapmodule.c to work on some documentation I
encountered this:

  24 #ifndef MS_WINDOWS
  25 #define UNIX
  26 #endif

 536 #ifdef MS_WINDOWS
 537         return PyInt_FromLong((long) FlushViewOfFile(self->data+offset, siz     e));
 538 #elif defined(UNIX)
 539         /* XXX semantics of return value? */
 540         /* XXX flags for msync? */
 541         if (-1 == msync(self->data + offset, size, MS_SYNC)) {
 542                 PyErr_SetFromErrno(mmap_module_error);
 543                 return NULL;
 544         }
 545         return PyInt_FromLong(0);
 546 #else
 547         PyErr_SetString(PyExc_ValueError, "flush not supported on this syst     em");
 548         return NULL;
 549 #endif

The #else will never be reached, unless I read it wrongly?

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
Once sent from the Golden Hall...

From asmodai at in-nomine.org  Wed Apr 16 15:29:15 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Wed, 16 Apr 2008 15:29:15 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <932f8baf0804160619q3446203ax40ee303145782c8f@mail.gmail.com>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
	<20080416125845.GG91329@nexus.in-nomine.org>
	<932f8baf0804160619q3446203ax40ee303145782c8f@mail.gmail.com>
Message-ID: <20080416132915.GI91329@nexus.in-nomine.org>

-On [20080416 15:20], Ralf Schmitt (schmir at gmail.com) wrote:
>I think documenting this insanity is just insane and doesn't help much.
>Anyway, you can now close http://bugs.python.org/issue2122 as wontfix.

Why?
I added documentation for the current way. I was not aware of the other
thread's mention of this issue. When your patch goes in --and looking at it
it makes sense to me-- I can easily adjust the documentation to reflect the
new state. :)

I can understand your apparent frustration at not having gotten someone to
look over your issue, but please: relax. I cannot make source changes, only
documentation.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
A kiss is a lovely trick designed by nature to stop speech when words
become superfluous...

From schmir at gmail.com  Wed Apr 16 15:35:34 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Wed, 16 Apr 2008 15:35:34 +0200
Subject: [Python-Dev] mmap.flush [was: NeedsReview keyword]
In-Reply-To: <20080416132915.GI91329@nexus.in-nomine.org>
References: <932f8baf0804151437k3ccf1ea6nb7b1adcaab75e71e@mail.gmail.com>
	<20080416125845.GG91329@nexus.in-nomine.org>
	<932f8baf0804160619q3446203ax40ee303145782c8f@mail.gmail.com>
	<20080416132915.GI91329@nexus.in-nomine.org>
Message-ID: <932f8baf0804160635x161e95a9yb85416c8b3106814@mail.gmail.com>

On Wed, Apr 16, 2008 at 3:29 PM, Jeroen Ruigrok van der Werven <
asmodai at in-nomine.org> wrote:

> -On [20080416 15:20], Ralf Schmitt (schmir at gmail.com) wrote:
> >I think documenting this insanity is just insane and doesn't help much.
> >Anyway, you can now close http://bugs.python.org/issue2122 as wontfix.
>
> Why?
> I added documentation for the current way. I was not aware of the other
> thread's mention of this issue. When your patch goes in --and looking at
> it
> it makes sense to me-- I can easily adjust the documentation to reflect
> the
> new state. :)
>
> I can understand your apparent frustration at not having gotten someone to
> look over your issue, but please: relax. I cannot make source changes,
> only
> documentation.
>

ok. makes sense. sorry, if I offended you.

Regards,
- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080416/ec87c425/attachment.htm 

From ijmorlan at cs.uwaterloo.ca  Wed Apr 16 15:37:28 2008
From: ijmorlan at cs.uwaterloo.ca (Isaac Morland)
Date: Wed, 16 Apr 2008 09:37:28 -0400 (EDT)
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>

On Wed, 16 Apr 2008, Paul Moore wrote:

> On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
>> What about the less confusing and more readily generalizable:
>>
>>   <range object 0, 1, ..., 9>
>>
>> It would also be helpful IMHO to use this kind of repr for most built-in
>> iterators and iterables, instead of their mosty-useless default repr.
>
> I quite like this. But as a non-beginner, I'll freely admit that my
> intuitions are suspect :-)

I like this too.  For iterators, though, would you always show the next 
couple of elements?  The values "contained in" the iterator will change as 
the iterator iterates.  Alternatively, the representation could be 
"frozen" to reflect the values originally pending in the iterator, but 
then the representation wouldn't show anything about the current state of 
the iterator.

Isaac Morland			CSCF Web Guru
DC 2554C, x36650		WWW Software Specialist

From wolever at cs.toronto.edu  Wed Apr 16 15:54:38 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Wed, 16 Apr 2008 09:54:38 -0400
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
	<Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
Message-ID: <9EC6ABE1-9BA8-4349-B8C3-F332E069AA83@cs.toronto.edu>

On 16-Apr-08, at 9:37 AM, Isaac Morland wrote:
> On Wed, 16 Apr 2008, Paul Moore wrote:
>> On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
>>> What about the less confusing and more readily generalizable:
>>>   <range object 0, 1, ..., 9>
>>>
>>> It would also be helpful IMHO to use this kind of repr for most  
>>> built-in
>>> iterators and iterables, instead of their mosty-useless default  
>>> repr.
>>
>> I quite like this. But as a non-beginner, I'll freely admit that my
>> intuitions are suspect :-)
>
> I like this too.  For iterators, though, would you always show the  
> next
> couple of elements?  The values "contained in" the iterator will  
> change as
> the iterator iterates.  Alternatively, the representation could be
> "frozen" to reflect the values originally pending in the iterator, but
> then the representation wouldn't show anything about the current  
> state of
> the iterator.
So would you mean something like:
<generator object <__main__.Foo instance at 0x835d0>, <__main__.Foo  
instance at 0x83620>, ...>
Or maybe:
<generator <__main__.Foo instance at 0x835d0>, <__main__.Foo instance  
at 0x83620>, ... >

While I agree in theory, I'm not sure I like the looks of it in  
practise.

From steve at holdenweb.com  Wed Apr 16 15:58:39 2008
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 16 Apr 2008 09:58:39 -0400
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>	<480523D2.2070705@canterbury.ac.nz>	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>	<480573D4.5040708@canterbury.ac.nz>	<20080416101544.GA10756@code0.codespeak.net>	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
	<Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
Message-ID: <fu50ms$6lv$1@ger.gmane.org>

Isaac Morland wrote:
> On Wed, 16 Apr 2008, Paul Moore wrote:
> 
>> On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
>>> What about the less confusing and more readily generalizable:
>>>
>>>   <range object 0, 1, ..., 9>
>>>
>>> It would also be helpful IMHO to use this kind of repr for most built-in
>>> iterators and iterables, instead of their mosty-useless default repr.
>> I quite like this. But as a non-beginner, I'll freely admit that my
>> intuitions are suspect :-)
> 
> I like this too.  For iterators, though, would you always show the next 
> couple of elements?  The values "contained in" the iterator will change as 
> the iterator iterates.  Alternatively, the representation could be 
> "frozen" to reflect the values originally pending in the iterator, but 
> then the representation wouldn't show anything about the current state of 
> the iterator.
> 
If you consume values from the iterator to display them in the repr() 
where do you then propose to store them until the application wants 
them, and how do you distinguish between "real" and "repr" consumption 
of the values? "The next couple of elements" is a dangerous thing to use 
unless you don't mind them disappearing. And the last one's right out - 
you;d end up storing lists anyway!

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ncoghlan at gmail.com  Wed Apr 16 16:03:46 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Apr 2008 00:03:46 +1000
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>	<480523D2.2070705@canterbury.ac.nz>	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>	<480573D4.5040708@canterbury.ac.nz>	<20080416101544.GA10756@code0.codespeak.net>	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
	<Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
Message-ID: <48060742.70101@gmail.com>

Isaac Morland wrote:
> On Wed, 16 Apr 2008, Paul Moore wrote:
> 
>> On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
>>> What about the less confusing and more readily generalizable:
>>>
>>>   <range object 0, 1, ..., 9>
>>>
>>> It would also be helpful IMHO to use this kind of repr for most built-in
>>> iterators and iterables, instead of their mosty-useless default repr.
>> I quite like this. But as a non-beginner, I'll freely admit that my
>> intuitions are suspect :-)
> 
> I like this too.  For iterators, though, would you always show the next 
> couple of elements?  The values "contained in" the iterator will change as 
> the iterator iterates.  Alternatively, the representation could be 
> "frozen" to reflect the values originally pending in the iterator, but 
> then the representation wouldn't show anything about the current state of 
> the iterator.

You can't write nice repr's for arbitrary iterators, or things like 
enumerate and friends that can operate on arbitrary iterator, because 
you have no way to get at the 'first few' and 'the last' value.

You can only have a nice representation like that for iterables like 
range which have random access to the underlying values. Note that range 
is NOT an iterator in Py3k as it has no __next__ method (the same hold 
trues for xrange in 2.x).

However, it would be really nice if collections.dict_keys, 
collections.dict_items and collections.dict_values could provide a nice 
repr implementation that was the rough equivalent of:

def full_repr(self):
   contents = ', '.join(map(repr, self))
   return "<%s: {%s}>" % (self.__class__.__name__, contents)

 >>> d = dict(a=1, b=2, c=3)
 >>> d
{'a': 1, 'c': 3, 'b': 2}
 >>> full_repr(d.keys())
"<dict_keys: {'a', 'c', 'b'}>"
 >>> full_repr(d.values())
'<dict_values: {1, 3, 2}>'
 >>> full_repr(d.items())
"<dict_items: {('a', 1), ('c', 3), ('b', 2)}>"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From bonelake at gmail.com  Wed Apr 16 16:11:27 2008
From: bonelake at gmail.com (Brad Miller)
Date: Wed, 16 Apr 2008 09:11:27 -0500
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <20080416101544.GA10756@code0.codespeak.net>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
Message-ID: <4CA553FE-4D6B-40D1-9517-D95756E3C1EC@gmail.com>


On Apr 16, 2008, at 5:15 AM, Armin Rigo wrote:

> Hi Greg,
>
> On Wed, Apr 16, 2008 at 03:34:44PM +1200, Greg Ewing wrote:
>>> That's why I proposed <0, 1, ..., 9> for repr(range(10)).
>>
>> My worry is that this will lead a newcomer into thinking
>> this is some kind of valid expression syntax.
>
> What about the less confusing and more readily generalizable:
>
>   <range object 0, 1, ..., 9>
>
> It would also be helpful IMHO to use this kind of repr for most  
> built-in
> iterators and iterables, instead of their mosty-useless default repr.
>

I think this is a great compromise.  It is much more helpful to the  
beginner than range(0,10).  This would also be a very simple change to  
the patch I already made :-)

I think it works nicely for the dict_keys, dict_values, and dict_items  
objects also as the student will see:

<dict_items object ('a', 1), ('b', 2), ... >

This reinforces that they will be iterating over tuples, which is much  
more helpful than a hexadecimal address.

Since ordering is not important for dictionary objects I wonder  
whether the ending value(s) are even needed or whether the first two  
are enough to help the student get an idea of what the object contains.

Brad


>
> A bientot,
>
> Armin.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/bonelake%40gmail.com


From curt at hagenlocher.org  Wed Apr 16 16:54:43 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 16 Apr 2008 07:54:43 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <fu50ms$6lv$1@ger.gmane.org>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
	<Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
	<fu50ms$6lv$1@ger.gmane.org>
Message-ID: <d2155e360804160754l24f06589k244c044b41935017@mail.gmail.com>

On Wed, Apr 16, 2008 at 6:58 AM, Steve Holden <steve at holdenweb.com> wrote:
>
> If you consume values from the iterator to display them in the repr()
> where do you then propose to store them until the application wants
> them, and how do you distinguish between "real" and "repr" consumption
> of the values? "The next couple of elements" is a dangerous thing to use
> unless you don't mind them disappearing.

Not only that, but you'd have no idea what the performance
consequences of accessing the "next" object might be.

--
Curt Hagenlocher
curt at hagenlocher.org

From guido at python.org  Wed Apr 16 16:57:06 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Apr 2008 07:57:06 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <20080416101544.GA10756@code0.codespeak.net>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
Message-ID: <ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>

On Wed, Apr 16, 2008 at 3:15 AM, Armin Rigo <arigo at tunes.org> wrote:
>  What about the less confusing and more readily generalizable:
>
>    <range object 0, 1, ..., 9>
>
>  It would also be helpful IMHO to use this kind of repr for most built-in
>  iterators and iterables, instead of their mosty-useless default repr.

We can't do this for iterators, since you can't print the values
without consuming the iterator. Printing something shouldn't have a
side effect on it. But for iterables (e.g. dict views) it should work
fine, and if others can agree with this I'd be happy to accept
patches.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ijmorlan at cs.uwaterloo.ca  Wed Apr 16 17:39:30 2008
From: ijmorlan at cs.uwaterloo.ca (Isaac Morland)
Date: Wed, 16 Apr 2008 11:39:30 -0400 (EDT)
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <9EC6ABE1-9BA8-4349-B8C3-F332E069AA83@cs.toronto.edu>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<79990c6b0804160559h2f63bdd0yef374fd6fe8dab7f@mail.gmail.com>
	<Pine.GSO.4.64.0804160930140.29553@core.cs.uwaterloo.ca>
	<9EC6ABE1-9BA8-4349-B8C3-F332E069AA83@cs.toronto.edu>
Message-ID: <Pine.GSO.4.64.0804161029010.29553@core.cs.uwaterloo.ca>

On Wed, 16 Apr 2008, David Wolever wrote:

> On 16-Apr-08, at 9:37 AM, Isaac Morland wrote:
>> On Wed, 16 Apr 2008, Paul Moore wrote:
>>> On 16/04/2008, Armin Rigo <arigo at tunes.org> wrote:
>>>> What about the less confusing and more readily generalizable:
>>>>  <range object 0, 1, ..., 9>
>>>> 
>>>> It would also be helpful IMHO to use this kind of repr for most built-in
>>>> iterators and iterables, instead of their mosty-useless default repr.
>>> 
>>> I quite like this. But as a non-beginner, I'll freely admit that my
>>> intuitions are suspect :-)
>> 
>> I like this too.  For iterators, though, would you always show the next
>> couple of elements?  The values "contained in" the iterator will change as
>> the iterator iterates.  Alternatively, the representation could be
>> "frozen" to reflect the values originally pending in the iterator, but
>> then the representation wouldn't show anything about the current state of
>> the iterator.
> So would you mean something like:
> <generator object <__main__.Foo instance at 0x835d0>, <__main__.Foo instance 
> at 0x83620>, ...>
> Or maybe:
> <generator <__main__.Foo instance at 0x835d0>, <__main__.Foo instance at 
> 0x83620>, ... >
>
> While I agree in theory, I'm not sure I like the looks of it in practise.

I was mostly responding to what I saw as a suggestion to change the 
representation of existing iterators.  It's been pointed out in a previous 
reply to my message that obtaining values from a general iterator for use 
in the representation is problematic at best, and in general I don't think 
it can be done in an acceptable fashion, because I can imagine code 
depending on values not being obtained from an iterator before they are 
explicitly requested by next().  We wouldn't want a call to __repr__() to 
change the operation of the iterator, so any idea to pull two values and 
store them somewhere isn't generally acceptable.

For some specific iterators which have access to all the underlying 
information, an informative representation is possible and frequently 
feasible.  My question simply concerned what it would look like.  Would it 
show the next couple of items, or the first couple of items (or something 
else)?

Isaac Morland			CSCF Web Guru
DC 2554C, x36650		WWW Software Specialist

From tnelson at onresolve.com  Wed Apr 16 19:51:53 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 16 Apr 2008 10:51:53 -0700
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and June
	21st-22nd.
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>


    Following on from the success of previous sprint/bugfix weekends and
    sprinting efforts at PyCon 2008, I'd like to propose the next two
    Global Python Sprint Weekends take place on the following dates:

        * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
        * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)

    It seems there are a few of the Python User Groups keen on meeting
    up in person and sprinting collaboratively, akin to PyCon, which I
    highly recommend.  I'd like to nominate Saturday across the board
    as the day for PUGs to meet up in person, with Sunday geared more
    towards an online collaboration day via IRC, where we can take care
    of all the little things that got in our way of coding on Saturday
    (like finalising/preparing/reviewing patches, updating tracker and
    documentation, writing tests ;-).

    For User Groups that are planning on meeting up to collaborate,
    please reply to this thread on python-dev at python.org and let every-
    one know your intentions!

    As is commonly the case, #python-dev on irc.freenode.net will be
    the place to be over the course of each sprint weekend; a large
    proportion of Python developers with commit access will be present,
    increasing the amount of eyes available to review and apply patches.

    For those that have an idea on areas they'd like to sprint on and
    want to look for other developers to rope in (or just to communicate
    plans in advance), please also feel free to jump on this thread via
    python-dev@ and indicate your intentions.

    For those that haven't the foggiest on what to work on, but would
    like to contribute, the bugs tracker at http://bugs.python.org is
    the best place to start.  Register an account and start searching
    for issues that you'd be able to lend a hand with.

    All contributors that submit code patches or documentation updates
    will typically get listed in Misc/ACKS.txt; come September when the
    final release of 2.6 and 3.0 come about, you'll be able to point at
    the tarball or .msi and exclaim loudly ``I helped build that!'',
    and actually back it up with hard evidence ;-)

    Bring on the pizza and Red Bull!

        Trent.

From fuzzyman at voidspace.org.uk  Wed Apr 16 20:40:44 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Apr 2008 19:40:44 +0100
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and
 June	21st-22nd.
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
Message-ID: <4806482C.4030305@voidspace.org.uk>

Trent Nelson wrote:
>     Following on from the success of previous sprint/bugfix weekends and
>     sprinting efforts at PyCon 2008, I'd like to propose the next two
>     Global Python Sprint Weekends take place on the following dates:
>
>         * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
>         * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)
>
>     It seems there are a few of the Python User Groups keen on meeting
>     up in person and sprinting collaboratively, akin to PyCon, which I
>     highly recommend.  I'd like to nominate Saturday across the board
>     as the day for PUGs to meet up in person, with Sunday geared more
>     towards an online collaboration day via IRC, where we can take care
>     of all the little things that got in our way of coding on Saturday
>     (like finalising/preparing/reviewing patches, updating tracker and
>     documentation, writing tests ;-).
>
>     For User Groups that are planning on meeting up to collaborate,
>     please reply to this thread on python-dev at python.org and let every-
>     one know your intentions!
>
>   

I should be able to help organise and attend the London contribution. 
Personally I'd like to work on the documentation changes / clean-up for 
the unittest module discussed recently.

Michael Foord


From tjreedy at udel.edu  Wed Apr 16 22:16:14 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Apr 2008 16:16:14 -0400
Subject: [Python-Dev] string representation of range in 3.0
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com><ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com><6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com><480523D2.2070705@canterbury.ac.nz><ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com><480573D4.5040708@canterbury.ac.nz><20080416101544.GA10756@code0.codespeak.net>
	<ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>
Message-ID: <fu5mqa$cd$1@ger.gmane.org>


"Guido van Rossum" <guido at python.org> wrote in message 
news:ca471dc20804160757u6a0257c8rff1ab5f68b5fc698 at mail.gmail.com...
| On Wed, Apr 16, 2008 at 3:15 AM, Armin Rigo <arigo at tunes.org> wrote:
| >  What about the less confusing and more readily generalizable:
| >
| >    <range object 0, 1, ..., 9>
| >
| >  It would also be helpful IMHO to use this kind of repr for most 
built-in
| >  iterators and iterables, instead of their mosty-useless default repr.
|
| We can't do this for iterators, since you can't print the values
| without consuming the iterator. Printing something shouldn't have a
| side effect on it. But for iterables (e.g. dict views) it should work
| fine, and if others can agree with this I'd be happy to accept
| patches.

Assuming we are discussing the output of str() rather that repr(),
I think we should give Armin's idea a try for iterables that don't get the 
full display
(as with lists, etc) but can harmlessly produce some values. 




From guido at python.org  Wed Apr 16 23:20:24 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Apr 2008 14:20:24 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <fu5mqa$cd$1@ger.gmane.org>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>
	<fu5mqa$cd$1@ger.gmane.org>
Message-ID: <ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>

On Wed, Apr 16, 2008 at 1:16 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>  "Guido van Rossum" <guido at python.org> wrote in message
>  news:ca471dc20804160757u6a0257c8rff1ab5f68b5fc698 at mail.gmail.com...
>
>
> | On Wed, Apr 16, 2008 at 3:15 AM, Armin Rigo <arigo at tunes.org> wrote:
>  | >  What about the less confusing and more readily generalizable:
>  | >
>  | >    <range object 0, 1, ..., 9>
>  | >
>  | >  It would also be helpful IMHO to use this kind of repr for most
>  built-in
>  | >  iterators and iterables, instead of their mosty-useless default repr.
>  |
>  | We can't do this for iterators, since you can't print the values
>  | without consuming the iterator. Printing something shouldn't have a
>  | side effect on it. But for iterables (e.g. dict views) it should work
>  | fine, and if others can agree with this I'd be happy to accept
>  | patches.
>
>  Assuming we are discussing the output of str() rather that repr(),
>  I think we should give Armin's idea a try for iterables that don't get the
>  full display
>  (as with lists, etc) but can harmlessly produce some values.

Why only str()? Note that the interactive prompt uses repr() to display values.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Wed Apr 16 23:24:20 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 16 Apr 2008 14:24:20 -0700
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
In-Reply-To: <ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
Message-ID: <52dc1c820804161424g121d76f5n7c144efc0770c804@mail.gmail.com>

On Mon, Apr 14, 2008 at 7:11 PM, Guido van Rossum <guido at python.org> wrote:

>
> But what operations raise EOFError? Surely you're not using
> raw_input()? It's really only there for teaching.
>
>
There are quite a few things in Lib/ that raise EOFError on their own.  Most
look like reasonable uses.  ftplib's getline() is likely the one raising it
such that it is ever seen when using urrlib2.  nntplib, telnetlib, xdrlib,
getpass, aifc, chunk, sunau, gzip, & mailbox also raise it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080416/99dc1e3b/attachment.htm 

From lists at cheimes.de  Wed Apr 16 23:39:40 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 16 Apr 2008 23:39:40 +0200
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>	<480523D2.2070705@canterbury.ac.nz>	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>	<480573D4.5040708@canterbury.ac.nz>	<20080416101544.GA10756@code0.codespeak.net>	<ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>	<fu5mqa$cd$1@ger.gmane.org>
	<ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>
Message-ID: <4806721C.9040407@cheimes.de>

Guido van Rossum schrieb:
> Why only str()? Note that the interactive prompt uses repr() to display values.

Does py3k still use the tp_print slot for the interactive prompt?

Christian

From guido at python.org  Thu Apr 17 00:30:12 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Apr 2008 15:30:12 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <4806721C.9040407@cheimes.de>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>
	<fu5mqa$cd$1@ger.gmane.org>
	<ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>
	<4806721C.9040407@cheimes.de>
Message-ID: <ca471dc20804161530w37171d61h4fd79bf5b04d4a08@mail.gmail.com>

No, tp_print is dead, unless I am terribly mistaken. (We didn't remove
the slot because that would require updating every single static type
initializer.)

On Wed, Apr 16, 2008 at 2:39 PM, Christian Heimes <lists at cheimes.de> wrote:
> Guido van Rossum schrieb:
>
> > Why only str()? Note that the interactive prompt uses repr() to display values.
>
>  Does py3k still use the tp_print slot for the interactive prompt?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Apr 17 02:27:43 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Apr 2008 12:27:43 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
Message-ID: <4806997F.50000@canterbury.ac.nz>

Guido van Rossum wrote:

> You are unique in demanding this.

Really? Nobody else wants a convenient way to
distinguish program bugs from exceptions caused
by external factors?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Apr 17 02:30:09 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Apr 2008 12:30:09 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
Message-ID: <48069A11.8060001@canterbury.ac.nz>

Guido van Rossum wrote:

> You are unique in demanding this.

I'm not asking for anything to be changed (I've
already agreed that EOFError can stay the way it
is), just pointing out what I think is a legitimate
use for custom EnvironmentError subclasses.

-- 
Greg

From amk at amk.ca  Thu Apr 17 03:51:10 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 16 Apr 2008 21:51:10 -0400
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and
	June 21st-22nd.
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
Message-ID: <20080417014450.GA8591@amk.local>

On Wed, Apr 16, 2008 at 10:51:53AM -0700, Trent Nelson wrote:
>     Following on from the success of previous sprint/bugfix weekends and
>     sprinting efforts at PyCon 2008, I'd like to propose the next two
>     Global Python Sprint Weekends take place on the following dates:

A great idea; thanks for organizing this!  I've updated the wiki page
at http://wiki.python.org/moin/PythonBugDay, so people can link to it,
and added a news item on www.python.org.

--amk

From bonelake at gmail.com  Thu Apr 17 03:57:32 2008
From: bonelake at gmail.com (Brad Miller)
Date: Wed, 16 Apr 2008 20:57:32 -0500
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804161530w37171d61h4fd79bf5b04d4a08@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com>
	<fu5mqa$cd$1@ger.gmane.org>
	<ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>
	<4806721C.9040407@cheimes.de>
	<ca471dc20804161530w37171d61h4fd79bf5b04d4a08@mail.gmail.com>
Message-ID: <84EB04A4-5443-4C7C-957F-43253CEF1368@gmail.com>

I worked up prototype implementation for dict_keys, dict_values, and  
dict_items

Here's an example of what the output looks like:

 >>> x = {chr(i):i for i in range(68,90)}
 >>> x.keys()
<dict_keys object 'E', 'D', 'G', ...>
 >>> x.values()
<dict_values object 69, 68, 71, ...>
 >>> x.items()
<dict_items object ('E', 69), ('D', 68), ('G', 71), ...>
 >>>

comments?

Are there other objects in this family that I should look at?

Brad

On Apr 16, 2008, at 5:30 PM, Guido van Rossum wrote:

> No, tp_print is dead, unless I am terribly mistaken. (We didn't remove
> the slot because that would require updating every single static type
> initializer.)
>
> On Wed, Apr 16, 2008 at 2:39 PM, Christian Heimes <lists at cheimes.de>  
> wrote:
>> Guido van Rossum schrieb:
>>
>>> Why only str()? Note that the interactive prompt uses repr() to  
>>> display values.
>>
>> Does py3k still use the tp_print slot for the interactive prompt?
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/bonelake%40gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080416/58f0230b/attachment-0001.htm 

From greg.ewing at canterbury.ac.nz  Thu Apr 17 04:03:11 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Apr 2008 14:03:11 +1200
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <4CA553FE-4D6B-40D1-9517-D95756E3C1EC@gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<20080416101544.GA10756@code0.codespeak.net>
	<4CA553FE-4D6B-40D1-9517-D95756E3C1EC@gmail.com>
Message-ID: <4806AFDF.6030308@canterbury.ac.nz>

Brad Miller wrote:

> <dict_items object ('a', 1), ('b', 2), ... >

I wouldn't include the word "object" in any of these.
Everything in Python is an object, so it's just
space-wasting noise.

-- 
Greg

From tjreedy at udel.edu  Thu Apr 17 04:49:02 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Apr 2008 22:49:02 -0400
Subject: [Python-Dev] string representation of range in 3.0
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com><ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com><6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com><480523D2.2070705@canterbury.ac.nz><ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com><480573D4.5040708@canterbury.ac.nz><20080416101544.GA10756@code0.codespeak.net><ca471dc20804160757u6a0257c8rff1ab5f68b5fc698@mail.gmail.com><fu5mqa$cd$1@ger.gmane.org>
	<ca471dc20804161420o5e51978csdf99d2427f328c5b@mail.gmail.com>
Message-ID: <fu6dqq$24o$1@ger.gmane.org>


"Guido van Rossum" <guido at python.org> wrote in message 
news:ca471dc20804161420o5e51978csdf99d2427f328c5b at mail.gmail.com...
| Why only str()? Note that the interactive prompt uses repr() to display 
values.

I was under the impression that for range, you still wanted to maintain the 
distinction between an evalable and a friendly representation.  If not, 
then ok, but I would then almost wonder whether to have both functions --  
until I remember the reason for two versions of string representation and 
your proposal (which I approve of) for both a full and abbreviated rep for 
classes.
| 




From tnelson at onresolve.com  Thu Apr 17 12:17:21 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 17 Apr 2008 03:17:21 -0700
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for 2.6/3.0
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>

Hi Barry,

Friendly poke to see if there's been any progress on acquiring the relevant certificates such that we can code sign binaries on Windows for 2.6/3.0 ;-)

        Trent.

From lists at cheimes.de  Thu Apr 17 13:13:07 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 17 Apr 2008 13:13:07 +0200
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
Message-ID: <480730C3.5010703@cheimes.de>

Trent Nelson schrieb:
> Hi Barry,
> 
> Friendly poke to see if there's been any progress on acquiring the relevant certificates such that we can code sign binaries on Windows for 2.6/3.0 ;-)

Can you explain:

 * Why the binaries should be signed?

 * What is required to sign the binaries?

 * Which binaries should be signed?

Christian

From tnelson at onresolve.com  Thu Apr 17 13:56:28 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 17 Apr 2008 04:56:28 -0700
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <480730C3.5010703@cheimes.de>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
	<480730C3.5010703@cheimes.de>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>


>  * Why the binaries should be signed?

    Makes the installation process on Windows Vista and Server 2008
    a little nicer; instead of getting an "unknown-executable-could-
    be-a-virus-aaaaaahhhh-watchout"-type dialog with a big red flag,
    you get a less threatening message saying that you're about to
    run something that's been digitally signed by the Python Software
    Foundation.  (I've come across a few entities (NSA, government
    bodies, etc), who mandate that all installers/binaries they get
    must be digitally signed.)

>  * What is required to sign the binaries?

 1. Obtain a code signing certificate from someone.  I used VeriSign.
    You end up with an .spc and a .pvk file.  You need to combine them
    into a single .pfx file via a tool called pvk2pfx.exe:

Usage:
    pvk2pfx -pvk <pvk-file> [-pi <pvk-pswd>] -spc <spc-file>
           [-pfx <pfx-file> [-po <pfx-pswd>] [-f]]

        -pvk <pvk-file>  - input PVK file name.
        -spc <spc-file>  - input SPC file name.
        -pfx <pfx-file>  - output PFX file name.
        -pi <pvk-pswd>   - PVK password.
        -po <pfx-pswd>   - PFX password; same as -pi if not given.
        -f               - force overwrite existing PFX file.

        if -pfx option is not given, an export wizard will pop up. in
        this case, options -po and -f are ignored.

C:\..> pvk2pfx.exe -pvk verisign-privatekey.pvk -pi ****** -spc onresolve-verisign.spc -po ****** -pfx onresolve-verisign.pfx

  3. The resulting .pfx file, onresolve-verisign.pfx in this case, can
     then be installed as a 'Personal' certificate in Windows, using the
     Certificate Management facility (CertMgr.exe).  When you install it,
     you provide a name that the certificate can be referred to by apps;
     in my case I just used 'VeriSign'.  This name is used below by the
     signtool.exe app.

  4. Sign the executable, MSI or DLL as follows:

C:\..> signtool.exe sign /i "VeriSign" /d "Python 2.6.0" /du http://www.python.org /t http://timestamp.verisign.com/scripts/timstamp.dll Python-2.6.msi
Successfully signed and timestamped: Python-2.6.msi

>  * Which binaries should be signed?

    Personally, once I figured out the steps above, I hooked the signing
    process into all my Visual Studio projects as a post-build step, such
    that I sign all .exe and .dll files.  Not really necessary, but eh,
    it does have the advantage of looking more professional (users can
    view properties on the .dll, for example, and see that it's been
    digitally signed by the PSF).  Additionally, it prevents any tampering;
    Windows can detect if it's been altered in any way since it's been
    signed, and will flat out prevent it from being loaded/run if that's
    the case.


        Trent.

From ziade.tarek at gmail.com  Thu Apr 17 15:44:23 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 17 Apr 2008 15:44:23 +0200
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and
	June 21st-22nd.
In-Reply-To: <4806482C.4030305@voidspace.org.uk>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
	<4806482C.4030305@voidspace.org.uk>
Message-ID: <94bdd2610804170644x4db329b0vf49de21a590c51e@mail.gmail.com>

On Wed, Apr 16, 2008 at 8:40 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Trent Nelson wrote:
>  >     Following on from the success of previous sprint/bugfix weekends and
>  >     sprinting efforts at PyCon 2008, I'd like to propose the next two
>  >     Global Python Sprint Weekends take place on the following dates:
>  >
>  >         * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
>  >         * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)
>  >
>  >     It seems there are a few of the Python User Groups keen on meeting
>  >     up in person and sprinting collaboratively, akin to PyCon, which I
>  >     highly recommend.  I'd like to nominate Saturday across the board
>  >     as the day for PUGs to meet up in person, with Sunday geared more
>  >     towards an online collaboration day via IRC, where we can take care
>  >     of all the little things that got in our way of coding on Saturday
>  >     (like finalising/preparing/reviewing patches, updating tracker and
>  >     documentation, writing tests ;-).
>  >
>  >     For User Groups that are planning on meeting up to collaborate,
>  >     please reply to this thread on python-dev at python.org and let every-
>  >     one know your intentions!
>  >
>  >
>
>  I should be able to help organise and attend the London contribution.
>  Personally I'd like to work on the documentation changes / clean-up for
>  the unittest module discussed recently.

We are trying to set up a team here in Paris,

Personnally I would like to continue the work started in distutils
(various patches)
and some friends here are interested in contributing on documentation.

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From aahz at pythoncraft.com  Thu Apr 17 16:25:34 2008
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 17 Apr 2008 07:25:34 -0700
Subject: [Python-Dev] string representation of range in 3.0
In-Reply-To: <ca471dc20804152145q6b5113e4v4263c39616c6a7f2@mail.gmail.com>
References: <9061B337-C6E0-4826-A5F3-846B2B60E1F8@gmail.com>
	<ca471dc20804071624j51e8a4dbu4df9e30e02ec5116@mail.gmail.com>
	<6D78AAC5-0C6F-4EA6-A491-AA0504BF0C6A@gmail.com>
	<480523D2.2070705@canterbury.ac.nz>
	<ca471dc20804151512t701e2516ge340a5e8238302e6@mail.gmail.com>
	<480573D4.5040708@canterbury.ac.nz>
	<ca471dc20804152145q6b5113e4v4263c39616c6a7f2@mail.gmail.com>
Message-ID: <20080417142534.GA17709@panix.com>

On Tue, Apr 15, 2008, Guido van Rossum wrote:
> On Tue, Apr 15, 2008 at 8:34 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Guido van Rossum wrote:
>>>
>>> That's why I proposed <0, 1, ..., 9> for repr(range(10)).
>>
>>  My worry is that this will lead a newcomer into thinking
>>  this is some kind of valid expression syntax.
> 
> You and Steven D'Aprano both.
> 
> So if this is indeed a bad idea, I'm stumped -- the 3.0 status quo is
> the best we can do, and teachers will just have to teach their
> students to write list(range(10)) if they want to see what it means.
> Or they can write an explicit for-loop (always useful on the first
> day):
> 
>   for i in range(10):
>     print i

No, they can't.  ;-)

(print is a function in 3.0....)

This may seem like a minor nit -- and it is, in some ways -- but I think
reminding people of the pedagogical issues involved in the transition is
important.  Granted, this is python-dev, so we should be using 2.x
syntax, but we're currently talking about 3.0 and that requires
clarifying what is meant/intended.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From fuzzyman at voidspace.org.uk  Thu Apr 17 15:49:15 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Apr 2008 14:49:15 +0100
Subject: [Python-Dev] Proposed unittest changes
Message-ID: <4807555B.2070206@voidspace.org.uk>

Hello all,

I'm starting to put together a list of cleanups (with documentation 
changes) for the unittest module. I thought someone had already done 
this but the issue with the most comments I could find was 2578 which 
doesn't go into much detail:

http://bugs.python.org/issue2578

The thing most people would like is test discovery - but that probably 
requires some discussion before anything can be added to unittest.

What I would like to do is PEP-8'ify the method names (widening the API 
rather than shrinking it!):

    assert_true
    assert_false
    assert_equals
    assert_not_equals
    assert_raises
    set_up
    tear_down
    failure_exception (? - class attribute)
    TestSuite.add_test

(etc)

Documenting that these are to be preferred to 'assertEquals' and 
'failUnlessEquals' (etc) and that the 'assert' statement should be used 
instead of 'assert_'.

Adding the following new asserts:

    assert_in    (member, container, msg=None)
    assert_not_in     (member, container, msg=None)
    assert_is     (first, second, msg=None)
    assert_not_is   (first, second, msg=None)
    assert_raises_with_message    (exc_class, message, callable, *args, 
**keywargs)

A decorator to turn a test function into a TestCase ("as_test_case" ?).

A simple 'RunTests' function that takes a collection of modules, test 
suites or test cases and runs them with TextTestRunner - passing on 
keyword arguments to the test runner. This makes running a test suite 
easier - once you have collected all your test cases together you can 
just pass them to this function so long as you are happy with the 
default runner (possibly allowing an alternative runner class to be 
provided as a keyword argument).

I would provide an implementation for discussion of course.

I would also like to make the error messages for "assert_equals" and 
"assert_not_equals" more useful - showing the objects that compare 
incorrectly even if an explicit message is passed in. Additionally, when 
comparing lists and tuples that are the same length show the members 
(and indices?) that were different.

I've copied Steve Purcell into this email, but his comments on issue 
2578 indicate that he is happy for 'us' to make changes and he no longer 
has a string sense of "ownership" of this module.

Michael Foord

From guido at python.org  Thu Apr 17 16:48:07 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 07:48:07 -0700
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
	<480730C3.5010703@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>
Message-ID: <ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>

This is going to be a major administrative hassle. Obviously you have
to be extremely cautious with the private part of the certificate, or
it's worthless. That means that there will probably be only one person
who can sign binaries. That person would have also have to build all
the binaries -- signing binaries you retrieved from the internet
sounds like a complete bypassing of the security procedures. *And*
that person would forever have to be extremely cautious with the
machine on which the certificate resides, keeping it turned off and
locked away securely most of the time, or else risk that the machine
is infected by malware, again bypassing the point. While the chances
of ever signing something bad are low, the bad effects could be huge
(sort of like the risk of an earthquake as compared to a car crash).

I'm not at all sure that we are set up to do this right, and that it
is worth the minor inconvenience to users of having to acknowledge a
red-flag dialog. After all, they will be ack'ing such dialogs all the
time if they are at all used to downloading software from the
internet.

--Guido

On Thu, Apr 17, 2008 at 4:56 AM, Trent Nelson <tnelson at onresolve.com> wrote:
>
>  >  * Why the binaries should be signed?
>
>     Makes the installation process on Windows Vista and Server 2008
>     a little nicer; instead of getting an "unknown-executable-could-
>     be-a-virus-aaaaaahhhh-watchout"-type dialog with a big red flag,
>     you get a less threatening message saying that you're about to
>     run something that's been digitally signed by the Python Software
>     Foundation.  (I've come across a few entities (NSA, government
>     bodies, etc), who mandate that all installers/binaries they get
>     must be digitally signed.)
>
>
>  >  * What is required to sign the binaries?
>
>   1. Obtain a code signing certificate from someone.  I used VeriSign.
>     You end up with an .spc and a .pvk file.  You need to combine them
>     into a single .pfx file via a tool called pvk2pfx.exe:
>
>  Usage:
>     pvk2pfx -pvk <pvk-file> [-pi <pvk-pswd>] -spc <spc-file>
>            [-pfx <pfx-file> [-po <pfx-pswd>] [-f]]
>
>         -pvk <pvk-file>  - input PVK file name.
>         -spc <spc-file>  - input SPC file name.
>         -pfx <pfx-file>  - output PFX file name.
>         -pi <pvk-pswd>   - PVK password.
>         -po <pfx-pswd>   - PFX password; same as -pi if not given.
>         -f               - force overwrite existing PFX file.
>
>         if -pfx option is not given, an export wizard will pop up. in
>         this case, options -po and -f are ignored.
>
>  C:\..> pvk2pfx.exe -pvk verisign-privatekey.pvk -pi ****** -spc onresolve-verisign.spc -po ****** -pfx onresolve-verisign.pfx
>
>   3. The resulting .pfx file, onresolve-verisign.pfx in this case, can
>      then be installed as a 'Personal' certificate in Windows, using the
>      Certificate Management facility (CertMgr.exe).  When you install it,
>      you provide a name that the certificate can be referred to by apps;
>      in my case I just used 'VeriSign'.  This name is used below by the
>      signtool.exe app.
>
>   4. Sign the executable, MSI or DLL as follows:
>
>  C:\..> signtool.exe sign /i "VeriSign" /d "Python 2.6.0" /du http://www.python.org /t http://timestamp.verisign.com/scripts/timstamp.dll Python-2.6.msi
>  Successfully signed and timestamped: Python-2.6.msi
>
>
>  >  * Which binaries should be signed?
>
>     Personally, once I figured out the steps above, I hooked the signing
>     process into all my Visual Studio projects as a post-build step, such
>     that I sign all .exe and .dll files.  Not really necessary, but eh,
>     it does have the advantage of looking more professional (users can
>     view properties on the .dll, for example, and see that it's been
>     digitally signed by the PSF).  Additionally, it prevents any tampering;
>     Windows can detect if it's been altered in any way since it's been
>     signed, and will flat out prevent it from being loaded/run if that's
>     the case.
>
>
>         Trent.
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 17 16:54:32 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 07:54:32 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4807555B.2070206@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
Message-ID: <ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>

I'm worried that a mass renaming would do anything but inconvenience
users during the already stressful 2->3 transition.

I'm more in favor of the original proposal of reducing the redundancy post-3.0.

If you're looking for useful features, Google has a set of extensions
to unittest.py that I find useful:

- module-level setUp and tearDown

- routines for comparing large lists and strings that produce useful
output indicating exactly where the inputs differ.

- assertLess etc.

- assertSameElements (sort of like assert(set(x) == set(y))

--Guido

On Thu, Apr 17, 2008 at 6:49 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Hello all,
>
>  I'm starting to put together a list of cleanups (with documentation
>  changes) for the unittest module. I thought someone had already done
>  this but the issue with the most comments I could find was 2578 which
>  doesn't go into much detail:
>
>  http://bugs.python.org/issue2578
>
>  The thing most people would like is test discovery - but that probably
>  requires some discussion before anything can be added to unittest.
>
>  What I would like to do is PEP-8'ify the method names (widening the API
>  rather than shrinking it!):
>
>     assert_true
>     assert_false
>     assert_equals
>     assert_not_equals
>     assert_raises
>     set_up
>     tear_down
>     failure_exception (? - class attribute)
>     TestSuite.add_test
>
>  (etc)
>
>  Documenting that these are to be preferred to 'assertEquals' and
>  'failUnlessEquals' (etc) and that the 'assert' statement should be used
>  instead of 'assert_'.
>
>  Adding the following new asserts:
>
>     assert_in    (member, container, msg=None)
>     assert_not_in     (member, container, msg=None)
>     assert_is     (first, second, msg=None)
>     assert_not_is   (first, second, msg=None)
>     assert_raises_with_message    (exc_class, message, callable, *args,
>  **keywargs)
>
>  A decorator to turn a test function into a TestCase ("as_test_case" ?).
>
>  A simple 'RunTests' function that takes a collection of modules, test
>  suites or test cases and runs them with TextTestRunner - passing on
>  keyword arguments to the test runner. This makes running a test suite
>  easier - once you have collected all your test cases together you can
>  just pass them to this function so long as you are happy with the
>  default runner (possibly allowing an alternative runner class to be
>  provided as a keyword argument).
>
>  I would provide an implementation for discussion of course.
>
>  I would also like to make the error messages for "assert_equals" and
>  "assert_not_equals" more useful - showing the objects that compare
>  incorrectly even if an explicit message is passed in. Additionally, when
>  comparing lists and tuples that are the same length show the members
>  (and indices?) that were different.
>
>  I've copied Steve Purcell into this email, but his comments on issue
>  2578 indicate that he is happy for 'us' to make changes and he no longer
>  has a string sense of "ownership" of this module.
>
>  Michael Foord
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Apr 17 17:05:20 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Apr 2008 11:05:20 -0400
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
	<480730C3.5010703@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>
	<ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>
Message-ID: <EDCD5108-671E-4A80-8A4C-123E7709F2FC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2008, at 10:48 AM, Guido van Rossum wrote:
> This is going to be a major administrative hassle. Obviously you have
> to be extremely cautious with the private part of the certificate, or
> it's worthless. That means that there will probably be only one person
> who can sign binaries. That person would have also have to build all
> the binaries -- signing binaries you retrieved from the internet
> sounds like a complete bypassing of the security procedures. *And*
> that person would forever have to be extremely cautious with the
> machine on which the certificate resides, keeping it turned off and
> locked away securely most of the time, or else risk that the machine
> is infected by malware, again bypassing the point. While the chances
> of ever signing something bad are low, the bad effects could be huge
> (sort of like the risk of an earthquake as compared to a car crash).
>
> I'm not at all sure that we are set up to do this right, and that it
> is worth the minor inconvenience to users of having to acknowledge a
> red-flag dialog. After all, they will be ack'ing such dialogs all the
> time if they are at all used to downloading software from the
> internet.

MvL is leading this effort and is currently trying to get a code  
signing certificate (my attempts having utterly failed on the Mac ;).   
Please coordinate with him.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBSAdnMHEjvBPtnXfVAQKkhQP/frCWs9sLcWyrAmDouCIq0n2X0B0TUbbG
+tEUQVBj2hJ/CVnmc6PjyFNjOhlfhIv+BI544QhsvoAlC2OdQe9d8hHX8sqnPUJy
lsm0gZ49ldqOhM91Q285RPtoELEEXQtfTYS2RUI/whNy+O/EDiRMVIDtNfaJOMgS
j6zeojy8d4E=
=m4aj
-----END PGP SIGNATURE-----

From fuzzyman at voidspace.org.uk  Thu Apr 17 16:59:01 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Apr 2008 15:59:01 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
Message-ID: <480765B5.2010904@voidspace.org.uk>

Guido van Rossum wrote:
> I'm worried that a mass renaming would do anything but inconvenience
> users during the already stressful 2->3 transition.
>
> I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>
>   

So nix the PEP-8'ifying until after 3.0.

So new methods should follow the current naming scheme (assertIn, 
assertNotIn etc).

> If you're looking for useful features, Google has a set of extensions
> to unittest.py that I find useful:
>
> - module-level setUp and tearDown
>   
So when a suite is made from a module the setUp should be called before 
the first test and tearDown after the last. I can look at that.

> - routines for comparing large lists and strings that produce useful
> output indicating exactly where the inputs differ.
>
> - assertLess etc.
>   
By etc I assume you mean:

    assertLessThan
    assertGreaterThan
    assertLessThanOrEquals
    assertGreaterThanOrEquals

Would not variants be useful as well - it seems not as the not of one is 
always another... (I think 'assertLessThan' reads better than 
'assertLess' but will do what I'm told...)

> - assertSameElements (sort of like assert(set(x) == set(y))
>   

Sounds good.

Did you look at the other proposals?

* Decorator to make a function a TestCase
* Convenience RunTests functions taking modules, suites and TestCases 
and running them
* Improved messages for assertEquals and assertNotEquals when an 
explicit message is passed in
* Improved message when comparing lists/tuples with assertEquals
* The additional asserts that I suggested (In/NotIn, RaisesWithMessage, 
Is/NotIs)

I think that there is still work I can do on the docs even before any 
grand renaming...

Michael Foord

> --Guido
>
> On Thu, Apr 17, 2008 at 6:49 AM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>   
>> Hello all,
>>
>>  I'm starting to put together a list of cleanups (with documentation
>>  changes) for the unittest module. I thought someone had already done
>>  this but the issue with the most comments I could find was 2578 which
>>  doesn't go into much detail:
>>
>>  http://bugs.python.org/issue2578
>>
>>  The thing most people would like is test discovery - but that probably
>>  requires some discussion before anything can be added to unittest.
>>
>>  What I would like to do is PEP-8'ify the method names (widening the API
>>  rather than shrinking it!):
>>
>>     assert_true
>>     assert_false
>>     assert_equals
>>     assert_not_equals
>>     assert_raises
>>     set_up
>>     tear_down
>>     failure_exception (? - class attribute)
>>     TestSuite.add_test
>>
>>  (etc)
>>
>>  Documenting that these are to be preferred to 'assertEquals' and
>>  'failUnlessEquals' (etc) and that the 'assert' statement should be used
>>  instead of 'assert_'.
>>
>>  Adding the following new asserts:
>>
>>     assert_in    (member, container, msg=None)
>>     assert_not_in     (member, container, msg=None)
>>     assert_is     (first, second, msg=None)
>>     assert_not_is   (first, second, msg=None)
>>     assert_raises_with_message    (exc_class, message, callable, *args,
>>  **keywargs)
>>
>>  A decorator to turn a test function into a TestCase ("as_test_case" ?).
>>
>>  A simple 'RunTests' function that takes a collection of modules, test
>>  suites or test cases and runs them with TextTestRunner - passing on
>>  keyword arguments to the test runner. This makes running a test suite
>>  easier - once you have collected all your test cases together you can
>>  just pass them to this function so long as you are happy with the
>>  default runner (possibly allowing an alternative runner class to be
>>  provided as a keyword argument).
>>
>>  I would provide an implementation for discussion of course.
>>
>>  I would also like to make the error messages for "assert_equals" and
>>  "assert_not_equals" more useful - showing the objects that compare
>>  incorrectly even if an explicit message is passed in. Additionally, when
>>  comparing lists and tuples that are the same length show the members
>>  (and indices?) that were different.
>>
>>  I've copied Steve Purcell into this email, but his comments on issue
>>  2578 indicate that he is happy for 'us' to make changes and he no longer
>>  has a string sense of "ownership" of this module.
>>
>>  Michael Foord
>>  _______________________________________________
>>  Python-Dev mailing list
>>  Python-Dev at python.org
>>  http://mail.python.org/mailman/listinfo/python-dev
>>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>     
>
>
>
>   


From jnoller at gmail.com  Thu Apr 17 17:19:35 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Apr 2008 11:19:35 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <4222a8490804170819g3f31590fo71ea23f30fce712d@mail.gmail.com>

On Thu, Apr 17, 2008 at 10:59 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Guido van Rossum wrote:
>  > I'm worried that a mass renaming would do anything but inconvenience
>  > users during the already stressful 2->3 transition.
>  >
>  > I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>  >
>  >
>
>  So nix the PEP-8'ifying until after 3.0.
>
>  So new methods should follow the current naming scheme (assertIn,
>  assertNotIn etc).
>
>
>  > If you're looking for useful features, Google has a set of extensions
>  > to unittest.py that I find useful:
>  >
>  > - module-level setUp and tearDown
>  >
>  So when a suite is made from a module the setUp should be called before
>  the first test and tearDown after the last. I can look at that.
>
>
>  > - routines for comparing large lists and strings that produce useful
>  > output indicating exactly where the inputs differ.
>  >
>  > - assertLess etc.
>  >
>  By etc I assume you mean:
>
>     assertLessThan
>     assertGreaterThan
>     assertLessThanOrEquals
>     assertGreaterThanOrEquals
>
>  Would not variants be useful as well - it seems not as the not of one is
>  always another... (I think 'assertLessThan' reads better than
>  'assertLess' but will do what I'm told...)
>
>
>  > - assertSameElements (sort of like assert(set(x) == set(y))
>  >
>
>  Sounds good.
>
>  Did you look at the other proposals?
>
>  * Decorator to make a function a TestCase
>  * Convenience RunTests functions taking modules, suites and TestCases
>  and running them
>  * Improved messages for assertEquals and assertNotEquals when an
>
> explicit message is passed in
>  * Improved message when comparing lists/tuples with assertEquals
>  * The additional asserts that I suggested (In/NotIn, RaisesWithMessage,
>  Is/NotIs)
>
>  I think that there is still work I can do on the docs even before any
>  grand renaming...
>
>  Michael Foord
>
>
>
>  > --Guido
>  >
>  > On Thu, Apr 17, 2008 at 6:49 AM, Michael Foord
>  > <fuzzyman at voidspace.org.uk> wrote:
>  >
>  >> Hello all,
>  >>
>  >>  I'm starting to put together a list of cleanups (with documentation
>  >>  changes) for the unittest module. I thought someone had already done
>  >>  this but the issue with the most comments I could find was 2578 which
>  >>  doesn't go into much detail:
>  >>
>  >>  http://bugs.python.org/issue2578
>  >>
>  >>  The thing most people would like is test discovery - but that probably
>  >>  requires some discussion before anything can be added to unittest.
>  >>
>  >>  What I would like to do is PEP-8'ify the method names (widening the API
>  >>  rather than shrinking it!):
>  >>
>  >>     assert_true
>  >>     assert_false
>  >>     assert_equals
>  >>     assert_not_equals
>  >>     assert_raises
>  >>     set_up
>  >>     tear_down
>  >>     failure_exception (? - class attribute)
>  >>     TestSuite.add_test
>  >>
>  >>  (etc)
>  >>
>  >>  Documenting that these are to be preferred to 'assertEquals' and
>  >>  'failUnlessEquals' (etc) and that the 'assert' statement should be used
>  >>  instead of 'assert_'.
>  >>
>  >>  Adding the following new asserts:
>  >>
>  >>     assert_in    (member, container, msg=None)
>  >>     assert_not_in     (member, container, msg=None)
>  >>     assert_is     (first, second, msg=None)
>  >>     assert_not_is   (first, second, msg=None)
>  >>     assert_raises_with_message    (exc_class, message, callable, *args,
>  >>  **keywargs)
>  >>
>  >>  A decorator to turn a test function into a TestCase ("as_test_case" ?).
>  >>
>  >>  A simple 'RunTests' function that takes a collection of modules, test
>  >>  suites or test cases and runs them with TextTestRunner - passing on
>  >>  keyword arguments to the test runner. This makes running a test suite
>  >>  easier - once you have collected all your test cases together you can
>  >>  just pass them to this function so long as you are happy with the
>  >>  default runner (possibly allowing an alternative runner class to be
>  >>  provided as a keyword argument).
>  >>
>  >>  I would provide an implementation for discussion of course.
>  >>
>  >>  I would also like to make the error messages for "assert_equals" and
>  >>  "assert_not_equals" more useful - showing the objects that compare
>  >>  incorrectly even if an explicit message is passed in. Additionally, when
>  >>  comparing lists and tuples that are the same length show the members
>  >>  (and indices?) that were different.
>  >>
>  >>  I've copied Steve Purcell into this email, but his comments on issue
>  >>  2578 indicate that he is happy for 'us' to make changes and he no longer
>  >>  has a string sense of "ownership" of this module.
>  >>
>  >>  Michael Foord

+1 on your changes Michael - I really like the decorator idea. Let me
know if you want help on adding the new stuff or updating the
documentation. What about aiming this at 2.6 instead of 3.0?

-jesse

From lists at cheimes.de  Thu Apr 17 17:27:19 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 17 Apr 2008 17:27:19 +0200
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <48076C57.5070601@cheimes.de>

Michael Foord schrieb:
> By etc I assume you mean:
> 
>     assertLessThan
>     assertGreaterThan
>     assertLessThanOrEquals
>     assertGreaterThanOrEquals
> 
> Would not variants be useful as well - it seems not as the not of one is 
> always another... (I think 'assertLessThan' reads better than 
> 'assertLess' but will do what I'm told...)

Most of the etc. could be simplified with a function assertOp which
takes an operator as first argument

import operator
def assertOp(self, op, a, b, msg):
    func = getattr(operator, op)
    self.assert_(func(a, b) ...)

assertOp("gt", a, b) == assert a > g

I also like to have some assert for is, type, isinstance, issubclass and
contains.

Christian

From steve at pythonconsulting.com  Thu Apr 17 17:39:16 2008
From: steve at pythonconsulting.com (Steve Purcell)
Date: Thu, 17 Apr 2008 17:39:16 +0200
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <2BC94709-57CD-4607-B4FC-1576A3285C7D@pythonconsulting.com>

Great that you're taking this on, Michael!


On 17 Apr 2008, at 16:59, Michael Foord wrote:

>> If you're looking for useful features, Google has a set of extensions
>> to unittest.py that I find useful:
>>
>> - module-level setUp and tearDown
>>
> So when a suite is made from a module the setUp should be called  
> before the first test and tearDown after the last. I can look at that.


Note that suites are just clumps of tests, and test runners can choose  
to run tests in any order, which might complicate the above.

In any case, I'd advise against per-module setUp/tearDown because it  
breaks the fundamental rule that any instance of TestCase can be  
safely run in isolation, secure in the knowledge that it will set  
itself up and tear itself down as required.  There are usually  
(always?) superior alternatives to module-level setUp, so I wouldn't  
suggest encouraging it by providing a sanctioned mechanism.

Also, I'd note that assert_() or similar may still be needed: it is  
provided as a preferred alternative to the assert() built-in because  
the latter stops working if you run Python with the -O switch, which  
might be a reasonable thing to do with a test suite.

Aside from these points, everything else looks great to me.  Better  
"diff-style" output for assertEquals of strings etc. has been lacking  
for ages, as well as collection-oriented assertions.

-Steve

P.S. Hi all!

From lists at cheimes.de  Thu Apr 17 17:32:14 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 17 Apr 2008 17:32:14 +0200
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <EDCD5108-671E-4A80-8A4C-123E7709F2FC@python.org>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>	<480730C3.5010703@cheimes.de>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>	<ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>
	<EDCD5108-671E-4A80-8A4C-123E7709F2FC@python.org>
Message-ID: <48076D7E.4060301@cheimes.de>

Barry Warsaw schrieb:
> MvL is leading this effort and is currently trying to get a code  
> signing certificate (my attempts having utterly failed on the Mac ;).   
> Please coordinate with him.

Apropos certificate, I like to suggest a PGP signing party for the next
PyCon. :]

Christian

From barry at python.org  Thu Apr 17 17:56:58 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 17 Apr 2008 11:56:58 -0400
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <48076D7E.4060301@cheimes.de>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>	<480730C3.5010703@cheimes.de>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>	<ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>
	<EDCD5108-671E-4A80-8A4C-123E7709F2FC@python.org>
	<48076D7E.4060301@cheimes.de>
Message-ID: <9C65BE76-8093-4790-B67F-3AD7F4769AEC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2008, at 11:32 AM, Christian Heimes wrote:
> Barry Warsaw schrieb:
>> MvL is leading this effort and is currently trying to get a code
>> signing certificate (my attempts having utterly failed on the Mac ;).
>> Please coordinate with him.
>
> Apropos certificate, I like to suggest a PGP signing party for the  
> next
> PyCon. :]

/me smacks his head for not doing one in 2008!

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBSAdzS3EjvBPtnXfVAQJEygP/SNB+nQ+XfEnOprAji9pGj5YcmJJSoKb3
A8UFR3LAI3EK3CVzgIXEeFuQOKZV34p0nTabVM9FEzsPrIbvnbzp1k4V4Go1bK/G
iBktzoG8F/VCmdbF7OjwBLb7O5GVVK1MRIO1KvdHp00lW311VYYMYdRuZ150lsct
doLzOH7kriU=
=LCTo
-----END PGP SIGNATURE-----

From fuzzyman at voidspace.org.uk  Thu Apr 17 18:11:32 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Apr 2008 17:11:32 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <48076C57.5070601@cheimes.de>
References: <4807555B.2070206@voidspace.org.uk>	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk> <48076C57.5070601@cheimes.de>
Message-ID: <480776B4.60503@voidspace.org.uk>

Christian Heimes wrote:
> Michael Foord schrieb:
>   
>> By etc I assume you mean:
>>
>>     assertLessThan
>>     assertGreaterThan
>>     assertLessThanOrEquals
>>     assertGreaterThanOrEquals
>>
>> Would not variants be useful as well - it seems not as the not of one is 
>> always another... (I think 'assertLessThan' reads better than 
>> 'assertLess' but will do what I'm told...)
>>     
>
> Most of the etc. could be simplified with a function assertOp which
> takes an operator as first argument
>
> import operator
> def assertOp(self, op, a, b, msg):
>     func = getattr(operator, op)
>     self.assert_(func(a, b) ...)
>
> assertOp("gt", a, b) == assert a > g
>
>   

Which choice would be up to consensus.

I'm agnostic on this one.

> I also like to have some assert for is, type, isinstance, issubclass and
> contains.
>
>   
is and contains ('in') I listed. type, isinstance, issubclass would be 
extra.

Michael


> Christian
>   


From fuzzyman at voidspace.org.uk  Thu Apr 17 18:20:49 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Apr 2008 17:20:49 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4222a8490804170819g3f31590fo71ea23f30fce712d@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>	
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>	
	<480765B5.2010904@voidspace.org.uk>
	<4222a8490804170819g3f31590fo71ea23f30fce712d@mail.gmail.com>
Message-ID: <480778E1.6060908@voidspace.org.uk>

Jesse Noller wrote:
> On Thu, Apr 17, 2008 at 10:59 AM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>   
>> Guido van Rossum wrote:
>>  > I'm worried that a mass renaming would do anything but inconvenience
>>  > users during the already stressful 2->3 transition.
>>  >
>>  > I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>>  >
>>  >
>>
>>  So nix the PEP-8'ifying until after 3.0.
>>
>>  So new methods should follow the current naming scheme (assertIn,
>>  assertNotIn etc).
>>
>> [snip...]
>>
>>     
>
> +1 on your changes Michael - I really like the decorator idea. Let me
> know if you want help on adding the new stuff or updating the
> documentation. What about aiming this at 2.6 instead of 3.0?
>
> -jesse
>   
 
As they will just be new features rather than API changes I would like 
to add these to 2.6 - although I think May 10th may be after the last 
alpha?

Michael

From rrr at ronadam.com  Thu Apr 17 18:38:57 2008
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 17 Apr 2008 11:38:57 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <48077D21.80001@ronadam.com>



Michael Foord wrote:
> Guido van Rossum wrote:
>> I'm worried that a mass renaming would do anything but inconvenience
>> users during the already stressful 2->3 transition.
>>
>> I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>>
>>   
> 
> So nix the PEP-8'ifying until after 3.0.
> 
> So new methods should follow the current naming scheme (assertIn, 
> assertNotIn etc).
> 
>> If you're looking for useful features, Google has a set of extensions
>> to unittest.py that I find useful:
>>
>> - module-level setUp and tearDown
>>   
> So when a suite is made from a module the setUp should be called before 
> the first test and tearDown after the last. I can look at that.
> 
>> - routines for comparing large lists and strings that produce useful
>> output indicating exactly where the inputs differ.
>>
>> - assertLess etc.
>>   
> By etc I assume you mean:
> 
>     assertLessThan
>     assertGreaterThan
>     assertLessThanOrEquals
>     assertGreaterThanOrEquals
> 
> Would not variants be useful as well - it seems not as the not of one is 
> always another... (I think 'assertLessThan' reads better than 
> 'assertLess' but will do what I'm told...)
> 
>> - assertSameElements (sort of like assert(set(x) == set(y))
>>   
> 
> Sounds good.
> 
> Did you look at the other proposals?
> 
> * Decorator to make a function a TestCase
> * Convenience RunTests functions taking modules, suites and TestCases 
> and running them
> * Improved messages for assertEquals and assertNotEquals when an 
> explicit message is passed in
> * Improved message when comparing lists/tuples with assertEquals
> * The additional asserts that I suggested (In/NotIn, RaisesWithMessage, 
> Is/NotIs)
> 
> I think that there is still work I can do on the docs even before any 
> grand renaming...


Something I found useful to improve messages is to have the tests catch and 
then report errors with it's own unique exceptions to indicate the failures.

What that does is make a very clear distinction between an error found in a 
module being tested vs test case.  An error in the test case results in a 
regular python exception and a trace back of the test case.  An error in 
the module being tested results in an specific unittest exception 
indicating what type of error was found in the module followed by the 
original exception and traceback from the module being tested.

Ron



class TestCase(unittest.TestCase):
     failureException = unittest.TestCase.failureException

     class Unexpected_Exception_Raised(failureException):
         def __init__(self, exc, ref):
             self.exc = exc
             self.ref = ref
         def __str__(self):
             return '\n'.join([repr(self.exc), '\nReference:', self.ref])

     class Wrong_Exception_Raised(Unexpected_Exception_Raised): pass

     class No_Exception_Raised(failureException):
         def __init__(self, result, ref=""):
             self.result = repr(result)
             self.ref = ref
         def __str__(self):
             return "returned -> " + '\n'.join([self.result,
                     '\nReference:', self.ref])

     class Wrong_Result_Returned(No_Exception_Raised):
         def __str__(self):
             return '\n'.join([self.result, '\nReference:', self.ref])

     def assertTestReturns(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             e0, e1, e2 = sys.exc_info()
             raise self.Unexpected_Exception_Raised, (e, ref), e2
         if result != expect:
             raise self.Wrong_Result_Returned(result, ref)

     def assertTestRaises(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             if isinstance(e, expect):
                 return e
             else:
                 e0, e1, e2 = sys.exc_info()
                 raise self.Wrong_Exception_Raised, (e, ref), e2
         raise self.No_Exception_Raised(result, ref)





From rrr at ronadam.com  Thu Apr 17 18:38:57 2008
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 17 Apr 2008 11:38:57 -0500
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <48077D21.80001@ronadam.com>



Michael Foord wrote:
> Guido van Rossum wrote:
>> I'm worried that a mass renaming would do anything but inconvenience
>> users during the already stressful 2->3 transition.
>>
>> I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>>
>>   
> 
> So nix the PEP-8'ifying until after 3.0.
> 
> So new methods should follow the current naming scheme (assertIn, 
> assertNotIn etc).
> 
>> If you're looking for useful features, Google has a set of extensions
>> to unittest.py that I find useful:
>>
>> - module-level setUp and tearDown
>>   
> So when a suite is made from a module the setUp should be called before 
> the first test and tearDown after the last. I can look at that.
> 
>> - routines for comparing large lists and strings that produce useful
>> output indicating exactly where the inputs differ.
>>
>> - assertLess etc.
>>   
> By etc I assume you mean:
> 
>     assertLessThan
>     assertGreaterThan
>     assertLessThanOrEquals
>     assertGreaterThanOrEquals
> 
> Would not variants be useful as well - it seems not as the not of one is 
> always another... (I think 'assertLessThan' reads better than 
> 'assertLess' but will do what I'm told...)
> 
>> - assertSameElements (sort of like assert(set(x) == set(y))
>>   
> 
> Sounds good.
> 
> Did you look at the other proposals?
> 
> * Decorator to make a function a TestCase
> * Convenience RunTests functions taking modules, suites and TestCases 
> and running them
> * Improved messages for assertEquals and assertNotEquals when an 
> explicit message is passed in
> * Improved message when comparing lists/tuples with assertEquals
> * The additional asserts that I suggested (In/NotIn, RaisesWithMessage, 
> Is/NotIs)
> 
> I think that there is still work I can do on the docs even before any 
> grand renaming...


Something I found useful to improve messages is to have the tests catch and 
then report errors with it's own unique exceptions to indicate the failures.

What that does is make a very clear distinction between an error found in a 
module being tested vs test case.  An error in the test case results in a 
regular python exception and a trace back of the test case.  An error in 
the module being tested results in an specific unittest exception 
indicating what type of error was found in the module followed by the 
original exception and traceback from the module being tested.

Ron



class TestCase(unittest.TestCase):
     failureException = unittest.TestCase.failureException

     class Unexpected_Exception_Raised(failureException):
         def __init__(self, exc, ref):
             self.exc = exc
             self.ref = ref
         def __str__(self):
             return '\n'.join([repr(self.exc), '\nReference:', self.ref])

     class Wrong_Exception_Raised(Unexpected_Exception_Raised): pass

     class No_Exception_Raised(failureException):
         def __init__(self, result, ref=""):
             self.result = repr(result)
             self.ref = ref
         def __str__(self):
             return "returned -> " + '\n'.join([self.result,
                     '\nReference:', self.ref])

     class Wrong_Result_Returned(No_Exception_Raised):
         def __str__(self):
             return '\n'.join([self.result, '\nReference:', self.ref])

     def assertTestReturns(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             e0, e1, e2 = sys.exc_info()
             raise self.Unexpected_Exception_Raised, (e, ref), e2
         if result != expect:
             raise self.Wrong_Result_Returned(result, ref)

     def assertTestRaises(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             if isinstance(e, expect):
                 return e
             else:
                 e0, e1, e2 = sys.exc_info()
                 raise self.Wrong_Exception_Raised, (e, ref), e2
         raise self.No_Exception_Raised(result, ref)





From ziade.tarek at gmail.com  Thu Apr 17 18:52:34 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 17 Apr 2008 18:52:34 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
Message-ID: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>

Hi,

shutil.copytree is very convenient to make recursive copies, but
os.walk has to be used everytime some filtering
has to be done on the files copied., if you want to avoid copying some files.

The code pattern with os.walk is pretty talkative :

--------------------- copying a source folder to a target folder, but
the pyc/pyo files
os.mkdir(target)
for root, dirs, filenames in os.walk(source):
    root_target = root.replace(source, target)
    for dir_ in dirs:
        target_dir = join(root_target, dir_)
        if os.path.exists(target_dir):
            continue
        os.mkdir(target_dir)
    for filename in filenames:
        filename_source = join(root, filename)
        filename_target = join(root_target, filename)
        if (os.path.exists(filename_target) or
            os.path.splitext(filename) in ('.pyc', '.pyo')):
            continue
        shutil.copyfile(filename_source, filename_target)
--------------------

If we could provide a callable argument to shutil.copytree, this would
allow simplifying a lot the code:

--------------------- copying a source to a target, but the pyc/pyo file
def filtering(source, target):
    return os.path.splitext(filename) not in ('.pyc', '.pyo')

shutil.copytree(source, target, filter_=filtering)
---------------------

This is a very current pattern in my code, and I think this could be
an interesting enhancement to
shutil. So if people think it is a good idea, I can work on a patch
and submit it to the tracker.

Regards

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From skip at pobox.com  Thu Apr 17 19:01:41 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 17 Apr 2008 12:01:41 -0500
Subject: [Python-Dev] Code signing of Windows .msi/.dll/.exe's for
	2.6/3.0
In-Reply-To: <9C65BE76-8093-4790-B67F-3AD7F4769AEC@python.org>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD422@EXMBX04.exchhosting.com>
	<480730C3.5010703@cheimes.de>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E23EAD433@EXMBX04.exchhosting.com>
	<ca471dc20804170748m437b2f68k4cb6198aca90cd9b@mail.gmail.com>
	<EDCD5108-671E-4A80-8A4C-123E7709F2FC@python.org>
	<48076D7E.4060301@cheimes.de>
	<9C65BE76-8093-4790-B67F-3AD7F4769AEC@python.org>
Message-ID: <18439.33397.636841.820546@montanaro-dyndns-org.local>


    >> Apropos certificate, I like to suggest a PGP signing party for the
    >> next PyCon. :]

    Barry> /me smacks his head for not doing one in 2008!

Maybe a bunch of Python local user groups could be coaxed into having
mini-signing parties during upcoming meetings, then let natural
cross-fertilization between groups (such as Fitz visiting Google Mountain
View or a few US folk attending EuroPython (mid-July), EuroSciPy (late July)
or PyCon UK (September)) connect those groups.

It wouldn't be as convenient as one big key signing party, but probably as
effective in the long run.

Skip

From guido at python.org  Thu Apr 17 19:01:51 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 10:01:51 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <480765B5.2010904@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <ca471dc20804171001n317c036bp9756e24382f4462f@mail.gmail.com>

On Thu, Apr 17, 2008 at 7:59 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>  By etc I assume you mean:
>
>    assertLessThan
>    assertGreaterThan
>    assertLessThanOrEquals
>    assertGreaterThanOrEquals

These names are used here:

assertListEqual(self, list1, list2, msg=None):
assertIn(self, a, b, msg=None):
assertNotIn(self, a, b, msg=None):
assertDictEqual(self, d1, d2, msg=None):
assertSameElements(self, expected_seq, actual_seq, msg=None):
assertMultiLineEqual(self, first, second, msg=None):
assertLess(self, a, b, msg=None):
assertLessEqual(self, a, b, msg=None):
assertGreater(self, a, b, msg=None):
assertGreaterEqual(self, a, b, msg=None):
assertCommandSucceeds(self, command):
assertCommandFails(self, command, regexes):

I can look into making those open source, but it'd probably more
efficient use of everyone's time to just reimplement them; most are
only a few lines. You can skip the assertCommand* ones, they're for
testing shell commands.


>  Did you look at the other proposals?

Not yet.

>  * Decorator to make a function a TestCase

But what about TOOWTDI?

>  * Convenience RunTests functions taking modules, suites and TestCases and
> running them

If it addresses the most common reason why people have to hack this
stuff, by all means.

>  * Improved messages for assertEquals and assertNotEquals when an explicit
> message is passed in

Sure.

>  * Improved message when comparing lists/tuples with assertEquals

I'd say do that in assertListEqual

>  * The additional asserts that I suggested (In/NotIn, RaisesWithMessage,
> Is/NotIs)

Sure. Google has In/NotIn

>  I think that there is still work I can do on the docs even before any grand
> renaming...

Go ahead!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ziade.tarek at gmail.com  Thu Apr 17 19:02:57 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 17 Apr 2008 19:02:57 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
Message-ID: <94bdd2610804171002q637bf590rda1caaa752d3a723@mail.gmail.com>

>  --------------------- copying a source to a target, but the pyc/pyo file
>  def filtering(source, target):
>     return os.path.splitext(filename) not in ('.pyc', '.pyo')
>
>  shutil.copytree(source, target, filter_=filtering)
>  ---------------------

oups, made a mistake in my example:

def filtering(source_path, target_path):
    return os.path.splitext(source_path) not in ('.pyc', '.pyo')

shutil.copytree(source, target, filter_=filtering)

From guido at python.org  Thu Apr 17 19:04:30 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 10:04:30 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <48076C57.5070601@cheimes.de>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk> <48076C57.5070601@cheimes.de>
Message-ID: <ca471dc20804171004uf27b875y4b3c1cdee005c206@mail.gmail.com>

On Thu, Apr 17, 2008 at 8:27 AM, Christian Heimes <lists at cheimes.de> wrote:
>  Most of the etc. could be simplified with a function assertOp which
>  takes an operator as first argument
>
>  import operator
>  def assertOp(self, op, a, b, msg):
>     func = getattr(operator, op)
>     self.assert_(func(a, b) ...)
>
>  assertOp("gt", a, b) == assert a > g

-1 on this; it requires more thinking and has more opportunities for
mistakes (e.g. why "gt" and not ">"?).

>  I also like to have some assert for is, type, isinstance, issubclass and
>  contains.

Yes. Michael had In/NotIn. I have needed all of the others too at various times!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 17 19:06:17 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 10:06:17 -0700
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
Message-ID: <ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>

Sounds like a neat little feature. Looking forward to it. Maybe the
most useful use case would be to provide glob-style patterns for
skipping files or directories (and their contents).

--Guido

On Thu, Apr 17, 2008 at 9:52 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi,
>
>  shutil.copytree is very convenient to make recursive copies, but
>  os.walk has to be used everytime some filtering
>  has to be done on the files copied., if you want to avoid copying some files.
>
>  The code pattern with os.walk is pretty talkative :
>
>  --------------------- copying a source folder to a target folder, but
>  the pyc/pyo files
>  os.mkdir(target)
>  for root, dirs, filenames in os.walk(source):
>     root_target = root.replace(source, target)
>     for dir_ in dirs:
>         target_dir = join(root_target, dir_)
>         if os.path.exists(target_dir):
>             continue
>         os.mkdir(target_dir)
>     for filename in filenames:
>         filename_source = join(root, filename)
>         filename_target = join(root_target, filename)
>         if (os.path.exists(filename_target) or
>             os.path.splitext(filename) in ('.pyc', '.pyo')):
>             continue
>         shutil.copyfile(filename_source, filename_target)
>  --------------------
>
>  If we could provide a callable argument to shutil.copytree, this would
>  allow simplifying a lot the code:
>
>  --------------------- copying a source to a target, but the pyc/pyo file
>  def filtering(source, target):
>     return os.path.splitext(filename) not in ('.pyc', '.pyo')
>
>  shutil.copytree(source, target, filter_=filtering)
>  ---------------------
>
>  This is a very current pattern in my code, and I think this could be
>  an interesting enhancement to
>  shutil. So if people think it is a good idea, I can work on a patch
>  and submit it to the tracker.
>
>  Regards
>
>  Tarek
>
>  --
>  Tarek Ziad? | Association AfPy | www.afpy.org
>  Blog FR | http://programmation-python.org
>  Blog EN | http://tarekziade.wordpress.com/
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gjcarneiro at gmail.com  Thu Apr 17 19:10:53 2008
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Thu, 17 Apr 2008 18:10:53 +0100
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
Message-ID: <a467ca4f0804171010v4886ba0fyb26f9ccf3268f9c1@mail.gmail.com>

On 17/04/2008, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>
> Hi,
>
> shutil.copytree is very convenient to make recursive copies, but
> os.walk has to be used everytime some filtering
> has to be done on the files copied., if you want to avoid copying some
> files.
>
> The code pattern with os.walk is pretty talkative :
>
> --------------------- copying a source folder to a target folder, but
> the pyc/pyo files
> os.mkdir(target)
> for root, dirs, filenames in os.walk(source):
>     root_target = root.replace(source, target)
>     for dir_ in dirs:
>         target_dir = join(root_target, dir_)
>         if os.path.exists(target_dir):
>             continue
>         os.mkdir(target_dir)
>     for filename in filenames:
>         filename_source = join(root, filename)
>         filename_target = join(root_target, filename)
>         if (os.path.exists(filename_target) or
>             os.path.splitext(filename) in ('.pyc', '.pyo')):
>             continue
>         shutil.copyfile(filename_source, filename_target)
> --------------------
>
> If we could provide a callable argument to shutil.copytree, this would
> allow simplifying a lot the code:
>
> --------------------- copying a source to a target, but the pyc/pyo file
> def filtering(source, target):
>     return os.path.splitext(filename) not in ('.pyc', '.pyo')
>
> shutil.copytree(source, target, filter_=filtering)
> ---------------------
>
> This is a very current pattern in my code, and I think this could be
> an interesting enhancement to
> shutil. So if people think it is a good idea, I can work on a patch
> and submit it to the tracker.


I also think this is a good idea; I recently was forced to copy-paste and
modify shutil.copytree into a project because of this limitation.

Regards
>
> Tarek
>
> --
>
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080417/ffb2ca5b/attachment.htm 

From tjreedy at udel.edu  Thu Apr 17 19:39:17 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 17 Apr 2008 13:39:17 -0400
Subject: [Python-Dev] Proposed unittest changes
References: <4807555B.2070206@voidspace.org.uk><ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk>
Message-ID: <fu8201$b07$1@ger.gmane.org>


"Michael Foord" <fuzzyman at voidspace.org.uk> wrote in message 
news:480765B5.2010904 at voidspace.org.uk...
| I think that there is still work I can do on the docs even before any
| grand renaming...

In a related thread, I proposed and Guido approved that the 2.6/3.0 docs
be changed to
1. specify that the Fail... names will disappear in the future
2. list the preferred Assert names first for each pair
3. use the preferred Assert names and only those names in the explanatory 
text and examples.

Have you or are you going to make these changes or should I open a separate 
tracker issue?

Terry




From asmodai at in-nomine.org  Thu Apr 17 22:46:29 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 17 Apr 2008 22:46:29 +0200
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <fu8201$b07$1@ger.gmane.org>
References: <480765B5.2010904@voidspace.org.uk> <fu8201$b07$1@ger.gmane.org>
Message-ID: <20080417204629.GS91329@nexus.in-nomine.org>

-On [20080417 19:46], Terry Reedy (tjreedy at udel.edu) wrote:
>Have you or are you going to make these changes or should I open a separate 
>tracker issue?

You can always put it on my account in the tracker (asmodai) and I'll get to
it in the coming days.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
The administration of justice is the firmest pillar of government...

From ziade.tarek at gmail.com  Fri Apr 18 00:27:06 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 18 Apr 2008 00:27:06 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
Message-ID: <94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>

On Thu, Apr 17, 2008 at 7:06 PM, Guido van Rossum <guido at python.org> wrote:
> Sounds like a neat little feature. Looking forward to it. Maybe the
>  most useful use case would be to provide glob-style patterns for
>  skipping files or directories (and their contents).

Alright I will work on it that way, thanks for the advice

Tarek

From jml at mumak.net  Fri Apr 18 00:28:17 2008
From: jml at mumak.net (Jonathan Lange)
Date: Fri, 18 Apr 2008 08:28:17 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
Message-ID: <d06a5cd30804171528o244c49came37d14cde500249d@mail.gmail.com>

On Fri, Apr 18, 2008 at 12:54 AM, Guido van Rossum <guido at python.org> wrote:
> I'm worried that a mass renaming would do anything but inconvenience
>  users during the already stressful 2->3 transition.
>
>  I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>
>  If you're looking for useful features, Google has a set of extensions
>  to unittest.py that I find useful:
>
>  - module-level setUp and tearDown
>
>  - routines for comparing large lists and strings that produce useful
>  output indicating exactly where the inputs differ.
>
>  - assertLess etc.
>
>  - assertSameElements (sort of like assert(set(x) == set(y))
>

Hi,

Some things that Bazaar, Twisted and Launchpad have found helpful:

- assertRaises returning the exception object that it catches. This
allows for easy testing of exception strings.

- Assertion methods for 'in', 'is' and 'isinstance' and their negations.

- TestCase.addCleanup. This method takes a function, args and kwargs
and pushes them to a stack of cleanups. Before tearDown is run, each
of these cleanups is popped off the stack and then run. This makes it
easier to acquire certain resources in tests:

e.g

def make_temp_dir(self):
    dir = tempfile.mkdtemp()
    self.addCleanup(shutil.rmtree, dir)
    return dir

Luckily, I have patches (with tests!) for these that I will be filing
on the tracker as soon as I get the opportunity.

jml

From jml at mumak.net  Fri Apr 18 00:31:15 2008
From: jml at mumak.net (Jonathan Lange)
Date: Fri, 18 Apr 2008 08:31:15 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4807555B.2070206@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
Message-ID: <d06a5cd30804171531h43d28f7fnf6dff20004a4682b@mail.gmail.com>

On Thu, Apr 17, 2008 at 11:49 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>     assert_raises_with_message    (exc_class, message, callable, *args,
>  **keywargs)
>

I don't think this one should go in.

I think it would be better if assertRaises just returned the exception
object that it catches. That way, you can test properties of the
exception other than its message.

jml

From guido at python.org  Fri Apr 18 00:34:53 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Apr 2008 15:34:53 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <d06a5cd30804171531h43d28f7fnf6dff20004a4682b@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>
	<d06a5cd30804171531h43d28f7fnf6dff20004a4682b@mail.gmail.com>
Message-ID: <ca471dc20804171534i6f23d71cp8398130c1945a24b@mail.gmail.com>

On Thu, Apr 17, 2008 at 3:31 PM, Jonathan Lange <jml at mumak.net> wrote:
> On Thu, Apr 17, 2008 at 11:49 PM, Michael Foord
>  <fuzzyman at voidspace.org.uk> wrote:
>  >     assert_raises_with_message    (exc_class, message, callable, *args,
>  >  **keywargs)
>  >
>
>  I don't think this one should go in.
>
>  I think it would be better if assertRaises just returned the exception
>  object that it catches. That way, you can test properties of the
>  exception other than its message.

Hm. I've got to say that returning the exception object is, um, an odd
API in the set of unittest APIs. I can see how it's sometimes more
powerful, but I'd say that in many cases assertRaisesWithMessage will
be easier to write and read. (And making it a regex match would be
even cooler.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jml at mumak.net  Fri Apr 18 00:40:21 2008
From: jml at mumak.net (Jonathan Lange)
Date: Fri, 18 Apr 2008 08:40:21 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ca471dc20804171534i6f23d71cp8398130c1945a24b@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>
	<d06a5cd30804171531h43d28f7fnf6dff20004a4682b@mail.gmail.com>
	<ca471dc20804171534i6f23d71cp8398130c1945a24b@mail.gmail.com>
Message-ID: <d06a5cd30804171540p222f5ba4u941f8faa39d3afc@mail.gmail.com>

On Fri, Apr 18, 2008 at 8:34 AM, Guido van Rossum <guido at python.org> wrote:
>
> On Thu, Apr 17, 2008 at 3:31 PM, Jonathan Lange <jml at mumak.net> wrote:
>  > On Thu, Apr 17, 2008 at 11:49 PM, Michael Foord
>  >  <fuzzyman at voidspace.org.uk> wrote:
>  >  >     assert_raises_with_message    (exc_class, message, callable, *args,
>  >  >  **keywargs)
>  >  >
>  >
>  >  I don't think this one should go in.
>  >
>  >  I think it would be better if assertRaises just returned the exception
>  >  object that it catches. That way, you can test properties of the
>  >  exception other than its message.
>
>  Hm. I've got to say that returning the exception object is, um, an odd
>  API in the set of unittest APIs. I can see how it's sometimes more
>  powerful, but I'd say that in many cases assertRaisesWithMessage will
>  be easier to write and read. (And making it a regex match would be
>  even cooler.)

I don't know about odd. It works and it's not obviously terrible.

Not having it the unittest API simply means that people who do want to
test non-message properties will rewrite assertRaises. Which is, in
fact, what we've already done.

jml

From steve at holdenweb.com  Fri Apr 18 00:53:51 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 17 Apr 2008 18:53:51 -0400
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ca471dc20804171534i6f23d71cp8398130c1945a24b@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>	<d06a5cd30804171531h43d28f7fnf6dff20004a4682b@mail.gmail.com>
	<ca471dc20804171534i6f23d71cp8398130c1945a24b@mail.gmail.com>
Message-ID: <fu8keg$h0a$1@ger.gmane.org>

Guido van Rossum wrote:
> On Thu, Apr 17, 2008 at 3:31 PM, Jonathan Lange <jml at mumak.net> wrote:
>> On Thu, Apr 17, 2008 at 11:49 PM, Michael Foord
>>  <fuzzyman at voidspace.org.uk> wrote:
>>  >     assert_raises_with_message    (exc_class, message, callable, *args,
>>  >  **keywargs)
>>  >
>>
>>  I don't think this one should go in.
>>
>>  I think it would be better if assertRaises just returned the exception
>>  object that it catches. That way, you can test properties of the
>>  exception other than its message.
> 
> Hm. I've got to say that returning the exception object is, um, an odd
> API in the set of unittest APIs. I can see how it's sometimes more
> powerful, but I'd say that in many cases assertRaisesWithMessage will
> be easier to write and read. (And making it a regex match would be
> even cooler.)
> 
In which case assertRaisesMatching (and then eventually 
assert_raises_matching) might be a better name for it?

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From fuzzyman at voidspace.org.uk  Fri Apr 18 00:54:18 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 17 Apr 2008 23:54:18 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <fu8201$b07$1@ger.gmane.org>
References: <4807555B.2070206@voidspace.org.uk><ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>	<480765B5.2010904@voidspace.org.uk>
	<fu8201$b07$1@ger.gmane.org>
Message-ID: <4807D51A.2060104@voidspace.org.uk>

Terry Reedy wrote:
> "Michael Foord" <fuzzyman at voidspace.org.uk> wrote in message 
> news:480765B5.2010904 at voidspace.org.uk...
> | I think that there is still work I can do on the docs even before any
> | grand renaming...
>
> In a related thread, I proposed and Guido approved that the 2.6/3.0 docs
> be changed to
> 1. specify that the Fail... names will disappear in the future
> 2. list the preferred Assert names first for each pair
> 3. use the preferred Assert names and only those names in the explanatory 
> text and examples.
>
> Have you or are you going to make these changes or should I open a separate 
> tracker issue?
>
>   

Yep - these were the doc changes I mentioned. I haven't started them but 
will unless someone beats me to it. :-)

Michael

> Terry
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


From tom at vector-seven.com  Fri Apr 18 01:40:06 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Fri, 18 Apr 2008 09:40:06 +1000
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and
 June 21st-22nd.
In-Reply-To: <94bdd2610804170644x4db329b0vf49de21a590c51e@mail.gmail.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>	<4806482C.4030305@voidspace.org.uk>
	<94bdd2610804170644x4db329b0vf49de21a590c51e@mail.gmail.com>
Message-ID: <4807DFD6.5040800@vector-seven.com>

Anybody in Melbourne keen for this? Not sure if I'll be able to make it 
myself, but I'd be interested to know if there's anybody in the area 
keen to do the sprint.

Cheers,
T

Tarek Ziad? wrote:
> On Wed, Apr 16, 2008 at 8:40 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>   
>> Trent Nelson wrote:
>>  >     Following on from the success of previous sprint/bugfix weekends and
>>  >     sprinting efforts at PyCon 2008, I'd like to propose the next two
>>  >     Global Python Sprint Weekends take place on the following dates:
>>  >
>>  >         * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
>>  >         * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)
>>  >
>>  >     It seems there are a few of the Python User Groups keen on meeting
>>  >     up in person and sprinting collaboratively, akin to PyCon, which I
>>  >     highly recommend.  I'd like to nominate Saturday across the board
>>  >     as the day for PUGs to meet up in person, with Sunday geared more
>>  >     towards an online collaboration day via IRC, where we can take care
>>  >     of all the little things that got in our way of coding on Saturday
>>  >     (like finalising/preparing/reviewing patches, updating tracker and
>>  >     documentation, writing tests ;-).
>>  >
>>  >     For User Groups that are planning on meeting up to collaborate,
>>  >     please reply to this thread on python-dev at python.org and let every-
>>  >     one know your intentions!
>>  >
>>  >
>>
>>  I should be able to help organise and attend the London contribution.
>>  Personally I'd like to work on the documentation changes / clean-up for
>>  the unittest module discussed recently.
>>     
>
> We are trying to set up a team here in Paris,
>
> Personnally I would like to continue the work started in distutils
> (various patches)
> and some friends here are interested in contributing on documentation.
>
> Tarek
>
>   


From musiccomposition at gmail.com  Fri Apr 18 04:24:29 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 17 Apr 2008 21:24:29 -0500
Subject: [Python-Dev] Interface to change Py3kWarning in Python
Message-ID: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>

I currently have a patch to make it possible to change py3k warnings
in Python through new functions in sys: issue 2458. I realize the
functions are rather ugly, but I don't think there is another
practical way to do it unless you want to be writing PySys_GetObject
and checking it for NULL whenever you add a Py3k warning.

-- 
Cheers,
Benjamin Peterson

From rbp at isnomore.net  Fri Apr 18 06:11:08 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Fri, 18 Apr 2008 01:11:08 -0300
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th
	and	June 21st-22nd.
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
Message-ID: <20080418041107.GD23236@isnomore.net>

On Wed, Apr 16 2008 at 02:51:53PM BRT, Trent Nelson <tnelson at onresolve.com> wrote:
> 
>     Following on from the success of previous sprint/bugfix weekends and
>     sprinting efforts at PyCon 2008, I'd like to propose the next two
>     Global Python Sprint Weekends take place on the following dates:
> 
>         * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
>         * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)
> 
>     It seems there are a few of the Python User Groups keen on meeting
>     up in person and sprinting collaboratively, akin to PyCon, which I
>     highly recommend.  I'd like to nominate Saturday across the board
>     as the day for PUGs to meet up in person, with Sunday geared more
>     towards an online collaboration day via IRC, where we can take care
>     of all the little things that got in our way of coding on Saturday
>     (like finalising/preparing/reviewing patches, updating tracker and
>     documentation, writing tests ;-).

The Sao Paulo User Group gathered "in person" to participate on the last bug
day, everyone enjoyed it and we're looking forward to the next one! I've
received strong support for our participation on May 10th/11th so we'll
definetly be there.


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From solipsis at pitrou.net  Fri Apr 18 11:13:05 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Apr 2008 09:13:05 +0000 (UTC)
Subject: [Python-Dev] thoughts on having EOFError inherit from
	EnvironmentError?
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
	<4806997F.50000@canterbury.ac.nz>
Message-ID: <loom.20080418T091149-177@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> Really? Nobody else wants a convenient way to
> distinguish program bugs from exceptions caused
> by external factors?

Why do you want to derive program bugs from EnvironmentError ? Usually I derive
them from ValueError, RuntimeError or simply Exception.




From lists at cheimes.de  Fri Apr 18 14:38:47 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 18 Apr 2008 14:38:47 +0200
Subject: [Python-Dev] Interface to change Py3kWarning in Python
In-Reply-To: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>
References: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>
Message-ID: <48089657.6010907@cheimes.de>

Benjamin Peterson schrieb:
> I currently have a patch to make it possible to change py3k warnings
> in Python through new functions in sys: issue 2458. I realize the
> functions are rather ugly, but I don't think there is another
> practical way to do it unless you want to be writing PySys_GetObject
> and checking it for NULL whenever you add a Py3k warning.

In Python we usually have to methods for the job, like
set_py3kwarningmode and get_py3kwarningmode. I don't like the enable*
and disable* style. Even the get/set methods are an ugly workaround for
the fact, Python doesn't support module properties. :/

How do you like a macro or function PyErr_Warn3k(msg) that does all the
dirty work?

Christian


From antonyjoseph89 at gmail.com  Fri Apr 18 15:41:56 2008
From: antonyjoseph89 at gmail.com (Antony Joseph)
Date: Fri, 18 Apr 2008 19:11:56 +0530
Subject: [Python-Dev] Need help in MAPI
Message-ID: <f14c4c340804180641p684ff556t50ab956fdcb8daa6@mail.gmail.com>

Hi,

My Code:
             mapi.MAPIInitialize(None)
            session = mapi.MAPILogonEx(0, MAPIProfile, None,
mapi.MAPI_EXTENDED | mapi.MAPI_USE_DEFAULT)

     I am  trying to send a mail  using the extended MAPI interface, I am
new  to work with MAPI.
I am trying to execute your code,i getting the following exception, and a
popup message of Either there is no default mail client or the current mail
client cannot fullfill the messaging request,please run Microsoftoffice
outlookand set it as the defaukt mail client.
I am using thunderbird as my default mail client , then i set my outlook as
my default mail client.its running fine.
can u tell me is there possiblites to run the code with out changing the
default mail client to Ms Outlook.

if could u find me a solution,that'd really helpfull.

Error:
C:\Documents and Settings\Administrator\Desktop>python mapisend.py
Traceback (most recent call last):
  File "mapisend.py", line 85, in <module>
    SendEMAPIMail(SendSubject, SendMessage, SendTo, MAPIProfile=MAPIProfile)
  File "mapisend.py", line 23, in SendEMAPIMail
    mapi.MAPIInitialize(None)
pywintypes.com_error: (-2147467259, 'Unspecified error', None, None)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080418/43cf6dc5/attachment.htm 

From asmodai at in-nomine.org  Fri Apr 18 16:12:16 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Fri, 18 Apr 2008 16:12:16 +0200
Subject: [Python-Dev] Known doctest bug with unicode?
Message-ID: <20080418141216.GF64048@nexus.in-nomine.org>

Is it a known doctest bug that when you have a dict with Unicode key values
that doctest dies with a KeyError?

When I excute my code from the regular python interpreter it works as
expected.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
Whenever you meet difficult situations dash forward bravely and joyfully...

From varmaa at gmail.com  Fri Apr 18 16:17:47 2008
From: varmaa at gmail.com (Atul Varma)
Date: Fri, 18 Apr 2008 07:17:47 -0700
Subject: [Python-Dev] Known doctest bug with unicode?
In-Reply-To: <20080418141216.GF64048@nexus.in-nomine.org>
References: <20080418141216.GF64048@nexus.in-nomine.org>
Message-ID: <361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>

Can you provide an example that fails?  This seems to work on my end, for
instance:

  >>> mydict = { u'\u2026' : 'ellipsis' }
  >>> mydict[u'\u2026']
  'ellipsis'

- Atul

On Fri, Apr 18, 2008 at 7:12 AM, Jeroen Ruigrok van der Werven <
asmodai at in-nomine.org> wrote:

> Is it a known doctest bug that when you have a dict with Unicode key
> values
> that doctest dies with a KeyError?
>
> When I excute my code from the regular python interpreter it works as
> expected.
>
> --
> Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
> $B%$%'%k!<%s(B $B%i%&%U%m%C%/(B $B%t%!%s(B $B%G%k(B $B%&%'%k%t%'%s(B
> http://www.in-nomine.org/ | http://www.rangaku.org/
> Whenever you meet difficult situations dash forward bravely and
> joyfully...
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/varmaa%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080418/25d0e2d8/attachment.htm 

From asmodai at in-nomine.org  Fri Apr 18 16:27:05 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Fri, 18 Apr 2008 16:27:05 +0200
Subject: [Python-Dev] Known doctest bug with unicode?
In-Reply-To: <361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>
References: <20080418141216.GF64048@nexus.in-nomine.org>
	<361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>
Message-ID: <20080418142705.GG64048@nexus.in-nomine.org>

# vim: set fileencoding=utf-8 :

kanamap = {
    u'?': 'a'
}

def transpose(word):
    """Convert a word in kana to its equivalent Hepburn romanisation.
    
    >>> transpose(u'?')
    'a'
    """
    transposed = ''
    for character in word:
        transposed += kanamap[character]
    return transposed

if __name__ == '__main__':
    import doctest
    doctest.testmod()

doctest:

[16:24] [ruigrok at akuma] (1) {20} % python trans.py
**********************************************************************
File "trans.py", line 11, in __main__.transpose
Failed example:
    transpose(u'?')
Exception raised:
    Traceback (most recent call last):
      File "doctest.py", line 1212, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__.transpose[0]>", line 1, in <module>
        transpose(u'?')
      File "trans.py", line 16, in transpose
        transposed += kanamap[character]
    KeyError: u'\xe3'
**********************************************************************
1 items had failures:
   1 of   1 in __main__.transpose
***Test Failed*** 1 failures.

normal interpreter:

>>> fromm trans import transpose
>>> transpose(u'?')
'a'

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/
They have learned nothing, and forgotten nothing...

From amauryfa at gmail.com  Fri Apr 18 16:32:42 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 18 Apr 2008 16:32:42 +0200
Subject: [Python-Dev] Need help in MAPI
In-Reply-To: <f14c4c340804180641p684ff556t50ab956fdcb8daa6@mail.gmail.com>
References: <f14c4c340804180641p684ff556t50ab956fdcb8daa6@mail.gmail.com>
Message-ID: <e27efe130804180732l2b5a4c96paebdddaacb1cd868@mail.gmail.com>

Hello,

Antony Joseph wrote:
> Hi,
>
> My Code:
>              mapi.MAPIInitialize(None)
>             session = mapi.MAPILogonEx(0, MAPIProfile, None,
> mapi.MAPI_EXTENDED | mapi.MAPI_USE_DEFAULT)
>
>       I am  trying to send a mail  using the extended MAPI interface, I am
> new  to work with MAPI.
>  I am trying to execute your code,i getting the following exception, and a
> popup message of Either there is no default mail client or the current mail
> client cannot fullfill the messaging request,please run Microsoftoffice
> outlookand set it as the defaukt mail client.
>  I am using thunderbird as my default mail client , then i set my outlook as
> my default mail client.its running fine.
>  can u tell me is there possiblites to run the code with out changing the
> default mail client to Ms Outlook.

First, this mailing list is for the development of the python language,
not for development with python.
Please ask this kind of questions on the comp.lang.python newsgroup.

Then, your problem is not related to python at all. The same call from
any other program would return the same error.
A quick google search gave:
http://www.tech-archive.net/Archive/Development/microsoft.public.win32.programmer.messaging/2007-04/msg00036.html
which describes the same problem (and a solution)

Otherwise you are welcome ;-)

-- 
Amaury Forgeot d'Arc

From rhamph at gmail.com  Fri Apr 18 18:05:19 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 18 Apr 2008 10:05:19 -0600
Subject: [Python-Dev] Known doctest bug with unicode?
In-Reply-To: <20080418142705.GG64048@nexus.in-nomine.org>
References: <20080418141216.GF64048@nexus.in-nomine.org>
	<361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>
	<20080418142705.GG64048@nexus.in-nomine.org>
Message-ID: <aac2c7cb0804180905u3fc0216eie659aed72a8e0311@mail.gmail.com>

On Fri, Apr 18, 2008 at 8:27 AM, Jeroen Ruigrok van der Werven
<asmodai at in-nomine.org> wrote:
> # vim: set fileencoding=utf-8 :
>
>  kanamap = {
>     u'$B$"(B': 'a'
>  }
>
>  def transpose(word):
>     """Convert a word in kana to its equivalent Hepburn romanisation.
>
>     >>> transpose(u'$B$"(B')
>     'a'
>     """
>     transposed = ''
>     for character in word:
>         transposed += kanamap[character]
>     return transposed
>
>  if __name__ == '__main__':
>     import doctest
>     doctest.testmod()
>
>  doctest:
>
>  [16:24] [ruigrok at akuma] (1) {20} % python trans.py
>  **********************************************************************
>  File "trans.py", line 11, in __main__.transpose
>  Failed example:
>     transpose(u'$B$"(B')
>  Exception raised:
>     Traceback (most recent call last):
>       File "doctest.py", line 1212, in __run
>         compileflags, 1) in test.globs
>       File "<doctest __main__.transpose[0]>", line 1, in <module>
>         transpose(u'$B$"(B')
>       File "trans.py", line 16, in transpose
>         transposed += kanamap[character]
>     KeyError: u'\xe3'
>  **********************************************************************
>  1 items had failures:
>    1 of   1 in __main__.transpose
>  ***Test Failed*** 1 failures.
>
>  normal interpreter:
>
>  >>> fromm trans import transpose
>  >>> transpose(u'$B$"(B')
>  'a'

What you've got is an 8-bit string containing a unicode literal.
Since this gets past the module's compilation stage, it doctest passes
it to the compiler again, and it defaults to iso-8859-1.  Thus
u'$B$"(B'.encode('utf-8').decode('latin-1') -> u'\xe3\x81\x82'.

Possible solutions:
1. Make the docstring itself unicode, assuming doctest allows this.
2. Call doctest explicitly, giving it the correct encoding.
3. See if you can put an encoding declaration in the doctest itself.
4. Make doctest smarter, so that it can grab the original module's encoding.
5. Wait until 3.0, where this is hopefully fixed by making doctests
use unicode by default?

-- 
Adam Olsen, aka Rhamphoryncus

From status at bugs.python.org  Fri Apr 18 18:06:37 2008
From: status at bugs.python.org (Tracker)
Date: Fri, 18 Apr 2008 18:06:37 +0200 (CEST)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20080418160637.1D64178351@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (04/11/08 - 04/18/08)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1839 open (+31) / 12641 closed ( +7) / 14480 total (+38)

Open issues with patches:   550

Average duration of open issues: 706 days.
Median duration of open issues: 1317 days.

Open Issues Breakdown
   open  1817 (+31)
pending    22 ( +0)

Issues Created Or Reopened (39)
_______________________________

Patch to emit "-J is reserved for Jython" on -J arg              04/11/08
CLOSED http://bugs.python.org/issue2617    created  fwierzbicki               
       patch                                                                   

Tile module: Add support for themed widgets                      04/11/08
       http://bugs.python.org/issue2618    created  wordtech                  
                                                                               

Document memoryview                                              04/11/08
       http://bugs.python.org/issue2619    created  benjamin.peterson         
                                                                               

Multiple buffer overflows in unicode processing                  04/11/08
       http://bugs.python.org/issue2620    created  jnferguson                
                                                                               

rename test_support to support                                   04/12/08
       http://bugs.python.org/issue2621    created  benjamin.peterson         
       patch, easy                                                             

Import errors in email.message.py                                04/12/08
       http://bugs.python.org/issue2622    created  JohnJackson               
                                                                               

Patch: xmlrpclib client ignores datetime tzinfo when creating is 04/12/08
       http://bugs.python.org/issue2623    created  lclark                    
       patch                                                                   

swig support in distutils should use the build and temp dirs     04/12/08
       http://bugs.python.org/issue2624    created  afflux                    
       patch                                                                   

mailbox.MH.get_message() treats result of get_sequences() as lis 04/12/08
       http://bugs.python.org/issue2625    created  hcs                       
                                                                               

If compile with gcc 4.3.0 python interpreter itself eats all mem 04/12/08
CLOSED http://bugs.python.org/issue2626    created  orivej                    
                                                                               

mark pgen generated files                                        04/13/08
CLOSED http://bugs.python.org/issue2627    created  benjamin.peterson         
       patch, easy                                                             

ftplib Persistent data connection                                04/13/08
       http://bugs.python.org/issue2628    created  jbell                     
       patch                                                                   

_Py_ForgetReference crash when called from _PyUnicode_New on Mem 04/14/08
       http://bugs.python.org/issue2629    created  gregory.p.smith           
                                                                               

repr() should not escape non-ASCII characters                    04/14/08
       http://bugs.python.org/issue2630    created  ishimoto                  
       patch                                                                   

IMPORT_NAME Documentation is incomplete                          04/14/08
       http://bugs.python.org/issue2631    created  pib                       
                                                                               

performance problem in socket._fileobject.read                   04/14/08
       http://bugs.python.org/issue2632    created  CurtHagenlocher           
       patch                                                                   

Improve subprocess.Popen() documentation ("env" parameter)       04/14/08
       http://bugs.python.org/issue2633    created  roysmith                  
                                                                               

os.execvpe() docs need to be more specific                       04/15/08
       http://bugs.python.org/issue2634    created  roysmith                  
                                                                               

textwrap: bug in 'fix_sentence_endings' option                   04/15/08
       http://bugs.python.org/issue2635    created  gscelsi                   
                                                                               

Regexp 2.6 (modifications to current re 2.2.2)                   04/15/08
       http://bugs.python.org/issue2636    created  timehorse                 
       patch                                                                   

urllib.quote() escapes characters unnecessarily and contrary to  04/15/08
       http://bugs.python.org/issue2637    created  tlesher                   
                                                                               

tkSimpleDialog Window Flashing                                   04/15/08
       http://bugs.python.org/issue2638    created  Longorh                   
                                                                               

shutil.copyfile() documentation is vague                         04/15/08
       http://bugs.python.org/issue2639    created  roysmith                  
                                                                               

"excel" csv option generates multiple lines                      04/15/08
CLOSED http://bugs.python.org/issue2640    created  laxrulz777                
                                                                               

setuptools gets site-packages wrong on Mac                       04/15/08
       http://bugs.python.org/issue2641    created  jorendorff                
                                                                               

MSVCRT packing in Windows Installer (3.0a4)                      04/16/08
       http://bugs.python.org/issue2642    created  wesley.spikes             
                                                                               

mmap_object_dealloc does not call FlushViewOfFile on windows     04/16/08
       http://bugs.python.org/issue2643    created  schmir                    
                                                                               

errors from msync ignored in mmap_object_dealloc                 04/16/08
       http://bugs.python.org/issue2644    created  schmir                    
                                                                               

httplib throws ValueError                                        04/16/08
       http://bugs.python.org/issue2645    created  niallo                    
       patch                                                                   

Python does not accept unicode keywords                          04/16/08
       http://bugs.python.org/issue2646    created  j5                        
       patch                                                                   

XML munges apos entity in tag content                            04/17/08
CLOSED http://bugs.python.org/issue2647    created  resplin                   
                                                                               

decimal receipe moneyfmt suppress leading 0                      04/17/08
CLOSED http://bugs.python.org/issue2648    created  cgrohmann                 
       patch                                                                   

poss. patch for fnmatch.py to add {.htm,html} style globbing     04/17/08
       http://bugs.python.org/issue2649    created  mark                      
                                                                               

re.escape should not escape underscore                           04/17/08
       http://bugs.python.org/issue2650    created  rsc                       
       easy                                                                    

Strings passed to KeyError do not round trip                     04/17/08
       http://bugs.python.org/issue2651    created  rharris                   
                                                                               

64 bit python  memory leak usage                                 04/17/08
       http://bugs.python.org/issue2652    created  kevin3210                 
                                                                               

string to float to int                                           04/18/08
CLOSED http://bugs.python.org/issue2653    created  wr                        
                                                                               

Typo at http://docs.python.org/dev/3.0/howto/doanddont.html      04/18/08
       http://bugs.python.org/issue2654    created  cartman                   
                                                                               

IOBaseError                                                      04/12/08
CLOSED http://bugs.python.org/issue1481036 reopened gregory.p.smith           
                                                                               



Issues Now Closed (28)
______________________

reduce tarfile memory footprint                                    64 days
       http://bugs.python.org/issue2058    lars.gustaebel            
       patch                                                                   

CSV Sniffer does not function properly on single column .csv fil   60 days
       http://bugs.python.org/issue2078    skip.montanaro            
       patch                                                                   

Patch to add a get_data function to pkgutil                        26 days
       http://bugs.python.org/issue2439    ncoghlan                  
       patch                                                                   

int and float accept bytes, complex does not                       22 days
       http://bugs.python.org/issue2483    marketdickinson           
                                                                               

Bazaar ignore file                                                 15 days
       http://bugs.python.org/issue2510    benjamin.peterson         
       patch, easy                                                             

Document IO module                                                 12 days
       http://bugs.python.org/issue2530    benjamin.peterson         
       patch                                                                   

atom sorting error when building ctypes                             9 days
       http://bugs.python.org/issue2559    theller                   
                                                                               

why is (default)dict so slow on tuples???                           1 days
       http://bugs.python.org/issue2607    rhettinger                
                                                                               

xml.dom.minidom documentation consistency and update                1 days
       http://bugs.python.org/issue2615    georg.brandl              
       patch                                                                   

ctypes.pointer(), ctypes.POINTER() speedup                          3 days
       http://bugs.python.org/issue2616    theller                   
       patch, patch                                                            

Patch to emit "-J is reserved for Jython" on -J arg                 1 days
       http://bugs.python.org/issue2617    fwierzbicki               
       patch                                                                   

If compile with gcc 4.3.0 python interpreter itself eats all mem    0 days
       http://bugs.python.org/issue2626    loewis                    
                                                                               

mark pgen generated files                                           0 days
       http://bugs.python.org/issue2627    benjamin.peterson         
       patch, easy                                                             

"excel" csv option generates multiple lines                         1 days
       http://bugs.python.org/issue2640    skip.montanaro            
                                                                               

XML munges apos entity in tag content                               0 days
       http://bugs.python.org/issue2647    loewis                    
                                                                               

decimal receipe moneyfmt suppress leading 0                         0 days
       http://bugs.python.org/issue2648    rhettinger                
       patch                                                                   

string to float to int                                              0 days
       http://bugs.python.org/issue2653    rhettinger                
                                                                               

^$ won't split on empty line                                     1595 days
       http://bugs.python.org/issue852532  mkc                       
                                                                               

Keyword or function to invoke pdb programmatically               1592 days
       http://bugs.python.org/issue853569  benjamin.peterson         
                                                                               

readline not implemented for UTF-16                              1483 days
       http://bugs.python.org/issue920680  benjamin.peterson         
                                                                               

overall index to distributed documentation                       1417 days
       http://bugs.python.org/issue960845  benjamin.peterson         
                                                                               

Memory leak in socket.py on Mac OS X                             1204 days
       http://bugs.python.org/issue1092502 gregory.p.smith           
                                                                               

cmd.Cmd().cmdloop() can't read from file                         1136 days
       http://bugs.python.org/issue1156280 georg.brandl              
                                                                               

IOBaseError                                                         1 days
       http://bugs.python.org/issue1481036 arigo                     
                                                                               

implement warnings module in C                                    460 days
       http://bugs.python.org/issue1631171 brett.cannon              
       patch                                                                   

object.__init__ shouldn't allow args/kwds                         393 days
       http://bugs.python.org/issue1683368 gvanrossum                
                                                                               

code that writes the PKG-INFO file doesnt handle unicode          330 days
       http://bugs.python.org/issue1721241 georg.brandl              
                                                                               

cvs.get_dialect() return a class object                           290 days
       http://bugs.python.org/issue1744580 skip.montanaro            
                                                                               



Top Issues Most Discussed (10)
______________________________

 19 Make range __eq__ work                                             9 days
open    http://bugs.python.org/issue2603   

 14 repr() should not escape non-ASCII characters                      4 days
open    http://bugs.python.org/issue2630   

 12 Multiple buffer overflows in unicode processing                    7 days
open    http://bugs.python.org/issue2620   

 11 string representation of range and dictionary views                8 days
open    http://bugs.python.org/issue2610   

 10 performance problem in socket._fileobject.read                     4 days
open    http://bugs.python.org/issue2632   

  9 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1          469 days
open    http://bugs.python.org/issue1628484

  8 [regression] reading from a urllib2 file descriptor happens byt   10 days
open    http://bugs.python.org/issue2601   

  6 Fast BytesIO implementation + misc changes                       102 days
open    http://bugs.python.org/issue1751   

  6 IOBaseError                                                        1 days
closed  http://bugs.python.org/issue1481036

  5 Import errors in email.message.py                                  6 days
open    http://bugs.python.org/issue2622   




From ncoghlan at gmail.com  Fri Apr 18 19:41:01 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2008 03:41:01 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4807555B.2070206@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
Message-ID: <4808DD2D.5090605@gmail.com>

Michael Foord wrote:
> that the 'assert' statement should be used 
> instead of 'assert_'.

assert statements are actually a bad idea in tests - they get skipped 
when the test suite is run with optimisation switched on.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From musiccomposition at gmail.com  Fri Apr 18 22:31:14 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 18 Apr 2008 15:31:14 -0500
Subject: [Python-Dev] Interface to change Py3kWarning in Python
In-Reply-To: <48089657.6010907@cheimes.de>
References: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>
	<48089657.6010907@cheimes.de>
Message-ID: <1afaf6160804181331q5ef5f41dvec83f6c74758fb24@mail.gmail.com>

On Fri, Apr 18, 2008 at 7:38 AM, Christian Heimes <lists at cheimes.de> wrote:
> Benjamin Peterson schrieb:
>
>
>
>  In Python we usually have to methods for the job, like
>  set_py3kwarningmode and get_py3kwarningmode. I don't like the enable*
>  and disable* style. Even the get/set methods are an ugly workaround for
>  the fact, Python doesn't support module properties. :/
I will look at that.
>
>  How do you like a macro or function PyErr_Warn3k(msg) that does all the
>  dirty work?
Sounds lovely.
>
>  Christian
>
>



-- 
Cheers,
Benjamin Peterson

From musiccomposition at gmail.com  Sat Apr 19 00:43:01 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 18 Apr 2008 17:43:01 -0500
Subject: [Python-Dev] unscriptable?
Message-ID: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>

Consider this error:
>>> 3["something"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is unsubscriptable

"unscriptable" seems rather ambiguous. How about "[object] cannot be indexed"?

-- 
Cheers,
Benjamin Peterson

From musiccomposition at gmail.com  Sat Apr 19 01:03:09 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 18 Apr 2008 18:03:09 -0500
Subject: [Python-Dev] unscriptable?
In-Reply-To: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
Message-ID: <1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>

On Fri, Apr 18, 2008 at 5:43 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> Consider this error:
>  >>> 3["something"]
>  Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>  TypeError: 'int' object is unsubscriptable
>
>  "unscriptable" seems rather ambiguous. How about "[object] cannot be indexed"?
Titus just noticed that I confused "unscriptable" with
"unsubscriptable."  :P Still, though, unsubscriptable seems to be a
Python invented word.
What does (un)subscriptable even mean?
>
>  --
>  Cheers,
>  Benjamin Peterson
>



-- 
Cheers,
Benjamin Peterson

From scott+python-dev at scottdial.com  Sat Apr 19 01:15:07 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 18 Apr 2008 19:15:07 -0400
Subject: [Python-Dev] unscriptable?
In-Reply-To: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
Message-ID: <48092B7B.1090503@scottdial.com>

Benjamin Peterson wrote:
> Consider this error:
>>>> 3["something"]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: 'int' object is unsubscriptable
> 
> "unscriptable" seems rather ambiguous. How about "[object] cannot be indexed"?
> 

Although I think this is a bit of bike-shedding, I would point out that 
CPython is itself inconsistent about this:

 >>> set([1,2,3])[0]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: 'set' object is unindexable

I tend to agree with Benjamin that "unsubscriptable" is a made-up word, 
but so is "unindexable". So, whatever.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From fuzzyman at voidspace.org.uk  Sat Apr 19 01:19:00 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 19 Apr 2008 00:19:00 +0100
Subject: [Python-Dev] unscriptable?
In-Reply-To: <48092B7B.1090503@scottdial.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<48092B7B.1090503@scottdial.com>
Message-ID: <48092C64.80209@voidspace.org.uk>

Scott Dial wrote:
> Benjamin Peterson wrote:
>   
>> Consider this error:
>>     
>>>>> 3["something"]
>>>>>           
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: 'int' object is unsubscriptable
>>
>> "unscriptable" seems rather ambiguous. How about "[object] cannot be indexed"?
>>
>>     
>
> Although I think this is a bit of bike-shedding, I would point out that 
> CPython is itself inconsistent about this:
>
>  >>> set([1,2,3])[0]
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: 'set' object is unindexable
>
> I tend to agree with Benjamin that "unsubscriptable" is a made-up word, 
> but so is "unindexable". So, whatever.
>   

It seems to be more common in the 'Python world' to refer to indexing 
than subscription and I admit that this error message has confused me in 
the past.

Michael

> -Scott
>
>   


From martin at v.loewis.de  Sat Apr 19 02:08:07 2008
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 19 Apr 2008 02:08:07 +0200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <48092B7B.1090503@scottdial.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<48092B7B.1090503@scottdial.com>
Message-ID: <480937E7.8050006@v.loewis.de>

> I tend to agree with Benjamin that "unsubscriptable" is a made-up word, 
> but so is "unindexable". So, whatever.

FWIW, here is what other tools give as error message in similar cases:

gcc: subscripted value is neither array nor pointer
g++: invalid types ?double[int]? for array subscript
javac: array required, but X found
ruby: undefined method `[]=' for 1:Fixnum (NoMethodError)

See also

http://tinyurl.com/4nn284

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Sat Apr 19 02:18:47 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Apr 2008 12:18:47 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <loom.20080418T091149-177@post.gmane.org>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
	<4806997F.50000@canterbury.ac.nz>
	<loom.20080418T091149-177@post.gmane.org>
Message-ID: <48093A67.5070702@canterbury.ac.nz>

Antoine Pitrou wrote:

> Why do you want to derive program bugs from EnvironmentError ? Usually I derive
> them from ValueError, RuntimeError or simply Exception.

I'm *not* talking about program bugs, I'm talking about
exceptions due to something the user did wrong.

I like to be able to do this:

   try:
     f = open(somefile)
     mungulate(f)
     f.close()
   except EnvironmentError, e:
     big_nasty_alert("Couldn't mungulate: %s" % e)

I *don't* want this to catch things like TypeError and
ValueError that indicate a program bug if they ever
escape. I want those to propagate and cause a traceback
or get handled at a higher level.

By deriving my own external-factors exceptions from
EnvironmentError, this all works very nicely.

 From its position in the exception hierarchy, this
seems to be just the sort of thing that EnvironmentError
is designed for, and I'm perplexed to be told that
it's not.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Apr 19 02:52:01 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Apr 2008 12:52:01 +1200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <48092C64.80209@voidspace.org.uk>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<48092B7B.1090503@scottdial.com> <48092C64.80209@voidspace.org.uk>
Message-ID: <48094231.8050607@canterbury.ac.nz>

Michael Foord wrote:
> It seems to be more common in the 'Python world' to refer to indexing 
> than subscription

Especially since the subscript isn't actually written
as a subscript, unless you have a particulary fancy
code editor...

-- 
Greg

From steve at pearwood.info  Sat Apr 19 04:12:55 2008
From: steve at pearwood.info (Steven)
Date: Sat, 19 Apr 2008 12:12:55 +1000
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <48093A67.5070702@canterbury.ac.nz>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
	<4806997F.50000@canterbury.ac.nz>
	<loom.20080418T091149-177@post.gmane.org>
	<48093A67.5070702@canterbury.ac.nz>
Message-ID: <20080419121255.150520e9.steve@pearwood.info>

On Sat, 19 Apr 2008 12:18:47 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Antoine Pitrou wrote:
> 
> > Why do you want to derive program bugs from EnvironmentError ? Usually I derive
> > them from ValueError, RuntimeError or simply Exception.
> 
> I'm *not* talking about program bugs, I'm talking about
> exceptions due to something the user did wrong.
> 
> I like to be able to do this:
> 
>    try:
>      f = open(somefile)
>      mungulate(f)
>      f.close()
>    except EnvironmentError, e:
>      big_nasty_alert("Couldn't mungulate: %s" % e)


It might help if you explain what sort of actual things that the user does 
wrong that you are talking about. From where I'm sitting, the only thing I 
can see that the user could do wrong is specify the wrong file. That doesn't
sound like an EnvironmentError to me, but I don't know that it should be a 
ValueError or TypeError either.



-- 
Steven <steve at pearwood.info>

From skip at pobox.com  Sat Apr 19 12:39:40 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 19 Apr 2008 05:39:40 -0500
Subject: [Python-Dev] test_subprocess failing
Message-ID: <18441.52204.719333.678164@montanaro-dyndns-org.local>

test_subprocess failed in Neal's regression test overnight.  My guess for
the culprit would be Christian's checkin on Friday:

  r62386 | christian.heimes | 2008-04-18 21:23:57 -0500 (Fri, 18 Apr 2008) | 2 lines

  Added kill, terminate and send_signal to subprocess.Popen.  The bits and
  pieces for the Windows side were already in place. The POSIX side is trivial
  (as usual) and uses os.kill().

Skip


From ncoghlan at gmail.com  Sat Apr 19 14:13:19 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Apr 2008 22:13:19 +1000
Subject: [Python-Dev] unscriptable?
In-Reply-To: <1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
Message-ID: <4809E1DF.9020306@gmail.com>

Benjamin Peterson wrote:
> On Fri, Apr 18, 2008 at 5:43 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> Consider this error:
>>  >>> 3["something"]
>>  Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>  TypeError: 'int' object is unsubscriptable
>>
>>  "unscriptable" seems rather ambiguous. How about "[object] cannot be indexed"?
> Titus just noticed that I confused "unscriptable" with
> "unsubscriptable."  :P Still, though, unsubscriptable seems to be a
> Python invented word.
> What does (un)subscriptable even mean?

You can't pass the object a subscript (the expression between the square 
brackets) and get a meaningful answer.

Being indexable is subtly different from being subscriptable - the 
former has stronger connotations of numeric indices and sequence-like 
behaviour (particularly since the introduction of operator.index), while 
the latter merely states that the container provides some kinds of 
mapping from subscripts to values in the container, without provide any 
implications as to the nature of that mapping.

'Unsubscriptable' may be a bit clumsy, but it's as accurate a 
description of the error as you're likely to find.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at pearwood.info  Sat Apr 19 15:05:08 2008
From: steve at pearwood.info (Steven)
Date: Sat, 19 Apr 2008 23:05:08 +1000
Subject: [Python-Dev] unscriptable?
In-Reply-To: <4809E1DF.9020306@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
Message-ID: <20080419230508.8468355a.steve@pearwood.info>

On Sat, 19 Apr 2008 22:13:19 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> Being indexable is subtly different from being subscriptable - the 
> former has stronger connotations of numeric indices and sequence-like 
> behaviour 

I dispute this. Indices aren't necessarily numeric (think of an A-Z 
file), and I don't believe you are correct about the connotations of
indexable being numeric or sequence-like. Think of index cards. 

With my maths background, I would expect that subscripts are (almost)
always numeric, e.g. x-subscript-i means the i-th x value, where i 
is a natural number.

+0 on keeping unsubscriptable.
+1 on changing it to unidexable.
-1 on keeping the current misspelling.


-- 
Steven D'Aprano

From g.brandl at gmx.net  Sat Apr 19 16:12:53 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 19 Apr 2008 16:12:53 +0200
Subject: [Python-Dev] Interface to change Py3kWarning in Python
In-Reply-To: <48089657.6010907@cheimes.de>
References: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>
	<48089657.6010907@cheimes.de>
Message-ID: <fucukh$jfh$1@ger.gmane.org>

Christian Heimes schrieb:
> Benjamin Peterson schrieb:
>> I currently have a patch to make it possible to change py3k warnings
>> in Python through new functions in sys: issue 2458. I realize the
>> functions are rather ugly, but I don't think there is another
>> practical way to do it unless you want to be writing PySys_GetObject
>> and checking it for NULL whenever you add a Py3k warning.
> 
> In Python we usually have to methods for the job, like
> set_py3kwarningmode and get_py3kwarningmode. I don't like the enable*
> and disable* style. Even the get/set methods are an ugly workaround for
> the fact, Python doesn't support module properties. :/
> 
> How do you like a macro or function PyErr_Warn3k(msg) that does all the
> dirty work?

Which additional dirty work would it do? It should never have to do more than
check a simple static C variable, for obvious reasons.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From musiccomposition at gmail.com  Sat Apr 19 17:37:36 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sat, 19 Apr 2008 10:37:36 -0500
Subject: [Python-Dev] Interface to change Py3kWarning in Python
In-Reply-To: <fucukh$jfh$1@ger.gmane.org>
References: <1afaf6160804171924g409c3770tcd4a9061fadeb258@mail.gmail.com>
	<48089657.6010907@cheimes.de> <fucukh$jfh$1@ger.gmane.org>
Message-ID: <1afaf6160804190837s1ac6ecfbxe6073cd813656942@mail.gmail.com>

On Sat, Apr 19, 2008 at 9:12 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Christian Heimes schrieb:
>  >
>  > How do you like a macro or function PyErr_Warn3k(msg) that does all the
>  > dirty work?
>
>  Which additional dirty work would it do? It should never have to do more than
>  check a simple static C variable, for obvious reasons.
Even if there's no dirty work, it could clean up this:

if (Py_Py3kWarningFlag &&
   PyErr_WarnEx(PyErr_DeprecationWarning, "msg", 2) < 0)

to

if (PyErr_WarnPy3k("msg", 2) < 0)
>
>  Georg



-- 
Cheers,
Benjamin Peterson

From lists at cheimes.de  Sat Apr 19 20:19:29 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 19 Apr 2008 20:19:29 +0200
Subject: [Python-Dev] test_subprocess failing
In-Reply-To: <18441.52204.719333.678164@montanaro-dyndns-org.local>
References: <18441.52204.719333.678164@montanaro-dyndns-org.local>
Message-ID: <480A37B1.8000006@cheimes.de>

skip at pobox.com schrieb:
> test_subprocess failed in Neal's regression test overnight.  My guess for
> the culprit would be Christian's checkin on Friday:
> 
>   r62386 | christian.heimes | 2008-04-18 21:23:57 -0500 (Fri, 18 Apr 2008) | 2 lines
> 
>   Added kill, terminate and send_signal to subprocess.Popen.  The bits and
>   pieces for the Windows side were already in place. The POSIX side is trivial
>   (as usual) and uses os.kill().

The processes were killed too early. The tests now wait until the
subprocess sends out a string.

Christian

From g.brandl at gmx.net  Sat Apr 19 23:23:36 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 19 Apr 2008 23:23:36 +0200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <20080419230508.8468355a.steve@pearwood.info>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>	<4809E1DF.9020306@gmail.com>
	<20080419230508.8468355a.steve@pearwood.info>
Message-ID: <fudns3$snk$2@ger.gmane.org>

Steven schrieb:
> On Sat, 19 Apr 2008 22:13:19 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> Being indexable is subtly different from being subscriptable - the 
>> former has stronger connotations of numeric indices and sequence-like 
>> behaviour 
> 
> I dispute this. Indices aren't necessarily numeric (think of an A-Z 
> file), and I don't believe you are correct about the connotations of
> indexable being numeric or sequence-like. Think of index cards. 

In normal language use, perhaps -- I'm no native speaker. But in the
Python world, Nick is right -- think of the __index__ special method that
adapts the object to a sequence index; it has to return an integer.

> With my maths background, I would expect that subscripts are (almost)
> always numeric, e.g. x-subscript-i means the i-th x value, where i 
> is a natural number.

Maths background isn't always directly transferable to computer
languages. Think 1 / 2 == 0 -- with terms, it's even worse.

An interesting point here is that the special methods are called
__getitem__ et al., smartly evading the problem. "'int' object
has no items" sounds equally strange to me though.

> +0 on keeping unsubscriptable.
> +1 on changing it to unidexable.
> -1 on keeping the current misspelling.

There is currently no misspelling.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Sat Apr 19 23:26:35 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 19 Apr 2008 23:26:35 +0200
Subject: [Python-Dev] 3k checkin mails to python-checkins
Message-ID: <fudo1l$snk$3@ger.gmane.org>

Since a few days, checkin notifications for the 3k branch seem to be sent
to both the python-checkins and the python-3000-checkins lists. Was that a
deliberate decision or has some bug crept into the SVN hook?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From jimjjewett at gmail.com  Sat Apr 19 23:31:12 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 19 Apr 2008 17:31:12 -0400
Subject: [Python-Dev] unscriptable?
Message-ID: <fb6fbf560804191431h720b79a6i79caf791ed45f08a@mail.gmail.com>

> I dispute this. Indices aren't necessarily numeric
> (think of an A-Z file),

Python has recently added an __index__ slot which means "as an
integer, and I really am an integer, I'm not just rounding like
int(3.4) would do"

So in the context of python, an index is numeric, whereas "subscript"
has already been used for hashtables with arbitrary keys.

-jJ

From stephen at xemacs.org  Sun Apr 20 01:03:17 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 20 Apr 2008 08:03:17 +0900
Subject: [Python-Dev] unscriptable?
In-Reply-To: <20080419230508.8468355a.steve@pearwood.info>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
	<20080419230508.8468355a.steve@pearwood.info>
Message-ID: <87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven writes:
 > On Sat, 19 Apr 2008 22:13:19 +1000
 > Nick Coghlan <ncoghlan at gmail.com> wrote:
 > 
 > > Being indexable is subtly different from being subscriptable - the 
 > > former has stronger connotations of numeric indices and sequence-like 
 > > behaviour 

 > With my maths background, I would expect that subscripts are (almost)
 > always numeric, e.g. x-subscript-i means the i-th x value, where i 
 > is a natural number.

I don't think you can get general agreement on which tend to connote
numerical indicies more.  On the other hand, in many contexts "index"
is a general concept, while "subscript" is a typographical convention
for representing an index.

+1 on indexable
+0.5 on subscriptable
-0.1 on scriptable

From scott+python-dev at scottdial.com  Sun Apr 20 01:00:29 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sat, 19 Apr 2008 19:00:29 -0400
Subject: [Python-Dev] unscriptable?
In-Reply-To: <fudns3$snk$2@ger.gmane.org>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>	<4809E1DF.9020306@gmail.com>	<20080419230508.8468355a.steve@pearwood.info>
	<fudns3$snk$2@ger.gmane.org>
Message-ID: <480A798D.5040806@scottdial.com>

Georg Brandl wrote:
> There is currently no misspelling.

We should at least make the set object match the others.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From greg.ewing at canterbury.ac.nz  Sun Apr 20 01:04:56 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Apr 2008 11:04:56 +1200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <4809E1DF.9020306@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
Message-ID: <480A7A98.8020407@canterbury.ac.nz>

Nick Coghlan wrote:

> Being indexable is subtly different from being subscriptable - the 
> former has stronger connotations of numeric indices and sequence-like 
> behaviour (particularly since the introduction of operator.index), while 
> the latter merely states that the container provides some kinds of 
> mapping from subscripts to values in the container, without provide any 
> implications as to the nature of that mapping.

I don't think everyone would agree with that. For example,
databases have these things that are called "indexes" even
when the value being used as a key isn't a number.

I've never really liked the term "subscript" in this
context, because to me it implies a typographical feature,
i.e. writing something below the baseline, which isn't
normally done in a textual programming language. The
term "index", on the other hand, directly suggests the
idea of looking something up, without relying on
indirection through typography and mathematical convention.

-- 
Greg

From ncoghlan at gmail.com  Sun Apr 20 03:03:21 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 20 Apr 2008 11:03:21 +1000
Subject: [Python-Dev] unscriptable?
In-Reply-To: <87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>	<4809E1DF.9020306@gmail.com>	<20080419230508.8468355a.steve@pearwood.info>
	<87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <480A9659.30704@gmail.com>

Stephen J. Turnbull wrote:
> Steven writes:
>  > On Sat, 19 Apr 2008 22:13:19 +1000
>  > Nick Coghlan <ncoghlan at gmail.com> wrote:
>  > 
>  > > Being indexable is subtly different from being subscriptable - the 
>  > > former has stronger connotations of numeric indices and sequence-like 
>  > > behaviour 
> 
>  > With my maths background, I would expect that subscripts are (almost)
>  > always numeric, e.g. x-subscript-i means the i-th x value, where i 
>  > is a natural number.
> 
> I don't think you can get general agreement on which tend to connote
> numerical indicies more.

Indices and arrays go hand-in-hand for me (probably due to my signal 
processing background) and I believe PEP 357's choice of __index__ for 
sequence indexing made that association reasonably official in Python terms.

>  On the other hand, in many contexts "index"
> is a general concept,  while "subscript" is a typographical convention
> for representing an index.

Granted, but the latter is explicitly used as the general term for 
sequence indexing and mapping in the relevant section of the language 
reference [1]. So specifically in the context of Python, the error 
message from 'int' is absolutely correct, and the error message from 
'set' isn't technically wrong either (although perhaps not quite as 
correct as it could be).

To suggest yet another colour for the bikeshed, maybe we should ditch 
both unindexable and unsubscriptable and go with "'int' is not a 
sequence or mapping". Any object which supports subscript notation will 
be one or the other.

Cheers,
Nick.

[1] http://docs.python.org/dev/reference/expressions.html#id7

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at holdenweb.com  Sun Apr 20 03:54:46 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 19 Apr 2008 21:54:46 -0400
Subject: [Python-Dev] unscriptable?
In-Reply-To: <480A9659.30704@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>	<4809E1DF.9020306@gmail.com>	<20080419230508.8468355a.steve@pearwood.info>	<87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480A9659.30704@gmail.com>
Message-ID: <fue7pu$tp$1@ger.gmane.org>

Nick Coghlan wrote:
[...]>
> To suggest yet another colour for the bikeshed, maybe we should ditch 
> both unindexable and unsubscriptable and go with "'int' is not a 
> sequence or mapping". Any object which supports subscript notation will 
> be one or the other.
> 
All the object needs to to to be "indexable" or "subscriptable" is to 
implement .__index__() or .__getitem__(). While sequences and mappings 
are the only built-in types to do so, this says nothing about 
user-defined types.

Couldn't we find a more obvious and direct error message like

"Illegal use of [] subscripting/indexing"?

one-more-coat-for-the-bikeshed-ly y'rs  - steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From stephen at xemacs.org  Sun Apr 20 04:33:57 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 20 Apr 2008 11:33:57 +0900
Subject: [Python-Dev] unscriptable?
In-Reply-To: <480A9659.30704@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
	<20080419230508.8468355a.steve@pearwood.info>
	<87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480A9659.30704@gmail.com>
Message-ID: <87wsmtcl0q.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > To suggest yet another colour for the bikeshed, maybe we should ditch 
 > both unindexable and unsubscriptable and go with "'int' is not a 
 > sequence or mapping".

I like this best.

From asmodai at in-nomine.org  Sun Apr 20 10:11:11 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sun, 20 Apr 2008 10:11:11 +0200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <480A9659.30704@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
	<20080419230508.8468355a.steve@pearwood.info>
	<87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp>
	<480A9659.30704@gmail.com>
Message-ID: <20080420081111.GO64048@nexus.in-nomine.org>

-On [20080420 03:04], Nick Coghlan (ncoghlan at gmail.com) wrote:
>To suggest yet another colour for the bikeshed, maybe we should ditch 
>both unindexable and unsubscriptable and go with "'int' is not a 
>sequence or mapping". Any object which supports subscript notation will 
>be one or the other.

Sounds even clearer to me. +1

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
It is necessary to be noble, and yet take humility as a basis. It is
necessary to be exalted, and yet take modesty as a foundation...

From martin at v.loewis.de  Sun Apr 20 11:25:28 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 20 Apr 2008 11:25:28 +0200
Subject: [Python-Dev] 3k checkin mails to python-checkins
In-Reply-To: <fudo1l$snk$3@ger.gmane.org>
References: <fudo1l$snk$3@ger.gmane.org>
Message-ID: <480B0C08.2010501@v.loewis.de>

> Since a few days, checkin notifications for the 3k branch seem to be sent
> to both the python-checkins and the python-3000-checkins lists. Was that a
> deliberate decision or has some bug crept into the SVN hook?

It's a bug, I'll look into it (when I find the time).

Regards,
Martin

From ziade.tarek at gmail.com  Mon Apr 21 00:15:57 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 21 Apr 2008 00:15:57 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
Message-ID: <94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>

I have submitted a patch for review here: http://bugs.python.org/issue2663

glob-style patterns or a callable (for complex cases) can be provided
to filter out files or directories.


Regards
Tarek

From steven.bethard at gmail.com  Mon Apr 21 02:25:34 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 20 Apr 2008 18:25:34 -0600
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
Message-ID: <d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>

On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> I have submitted a patch for review here: http://bugs.python.org/issue2663
>
>  glob-style patterns or a callable (for complex cases) can be provided
>  to filter out files or directories.

I'm not a big fan of the sequence-or-callable argument. Why not just
make it a callable argument, and supply a utility function so that you
can write something like::

    exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
    shutil.copytree(src_dir, dst_dir, exclude=exclude_func)

?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From ijmorlan at cs.uwaterloo.ca  Mon Apr 21 02:38:03 2008
From: ijmorlan at cs.uwaterloo.ca (Isaac Morland)
Date: Sun, 20 Apr 2008 20:38:03 -0400 (EDT)
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0804202030520.19492@core.cs.uwaterloo.ca>

On Sun, 20 Apr 2008, Steven Bethard wrote:

> On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> I have submitted a patch for review here: http://bugs.python.org/issue2663
>>
>>  glob-style patterns or a callable (for complex cases) can be provided
>>  to filter out files or directories.
>
> I'm not a big fan of the sequence-or-callable argument. Why not just
> make it a callable argument, and supply a utility function so that you
> can write something like::
>
>    exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
>    shutil.copytree(src_dir, dst_dir, exclude=exclude_func)

Even if a glob pattern filter is considered useful enough to be worth 
special-casing, the glob capability should also be exposed via something 
like your excluding_patterns constructor and additionally as a function 
that can be called by another function intended for use as a callable 
argument.

If it is not, then doing something like "files matching these glob 
patterns except for those matching this non-glob-expressible condition and 
also those files matching this second non-glob-expressible condition" 
becomes painful because the glob part essentially needs to be 
re-implemented.

Isaac Morland			CSCF Web Guru
DC 2554C, x36650		WWW Software Specialist

From guido at python.org  Mon Apr 21 06:33:25 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 20 Apr 2008 21:33:25 -0700
Subject: [Python-Dev] unscriptable?
In-Reply-To: <480A9659.30704@gmail.com>
References: <1afaf6160804181543g63e5b5acvc5c675b42a3111fc@mail.gmail.com>
	<1afaf6160804181603x1c5ecfcagcac4534321a37c6c@mail.gmail.com>
	<4809E1DF.9020306@gmail.com>
	<20080419230508.8468355a.steve@pearwood.info>
	<87zlrpcuru.fsf@uwakimon.sk.tsukuba.ac.jp> <480A9659.30704@gmail.com>
Message-ID: <ca471dc20804202133g4a1e2a5dg5f3ebb00b2eb9613@mail.gmail.com>

On Sat, Apr 19, 2008 at 6:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>  To suggest yet another colour for the bikeshed, maybe we should ditch
>  both unindexable and unsubscriptable and go with "'int' is not a
>  sequence or mapping". Any object which supports subscript notation will
>  be one or the other.

I wouldn't bet my life on that. __getitem__ overloading for all sorts
of nefarious purposes has quite a following. I'd prefer a message that
doesn't get into what x "is" but sticks to the point at hand, which is
that it doesn't support __getitem__.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ziade.tarek at gmail.com  Mon Apr 21 09:48:47 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 21 Apr 2008 09:48:47 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <Pine.GSO.4.64.0804202030520.19492@core.cs.uwaterloo.ca>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
	<Pine.GSO.4.64.0804202030520.19492@core.cs.uwaterloo.ca>
Message-ID: <94bdd2610804210048x1afc7b27j5598d5cac0ae3336@mail.gmail.com>

The pattern matching uses the src_dir to call glob.glob(), which returns
the list of files to be excluded. That's why I added within the
copytree() function.

To make an excluding_patterns work, it could be coded like this::

    def excluding_patterns(*patterns):
        def _excluding_patterns(filepath):
            exclude_files = []
            dir_ = os.path.dirname(filepath)
            for pattern in patterns:
                pattern = os.path.join(dir_, pattern)
                exclude_files.extend(glob.glob(pattern))
            return path in exclude_files
        return _excluding_patterns

But I can see some performance issues, as the glob function will
be called within the loop to test each file or folder::

    def copytree(src, dst, exclude):
        ...
        for name in names:
            srcname = os.path.join(src, name)
            if exclude(srcname):
                continue
            ...
        ...

Adding it at the beginning of the `copytree`  function would then
be better for performance, but means that the callable has to return
a list of matching files instead of the match result itself::

    def excluding_patterns(*patterns):
        def _excluding_patterns(path):
            exclude_files = []
            for pattern in patterns:
                pattern = os.path.join(dir_, pattern)
                exclude_files.extend(glob.glob(pattern))
            return exclude_files

Then in copytree::

    def copytree(src, dst, exclude):
        ...
        excluded = exclude(src)
        ...
        for name in names:
            srcname = os.path.join(src, name)
            if srcname in excluded:
                continue
            ...
        ...

But this means that people that wants to implement their own
callable will have to provide a function that returns a list
of excluded files, therefore they won't be free to implement
what they want.

We could have two parameters, one for the glob-style sequence
and one for the callable, to be able to use them at the
appropriate places in the function, but I think this would
make the function signature rather heavy::

    def copytree(src, dst, exclude_patterns=None, exclude_function=None):
        ...


That's why I would be in favor of sequence-or-callable argument
even if I admit that it is not the pretiest way to present
an argument.

Regards

Tarek

On Mon, Apr 21, 2008 at 2:38 AM, Isaac Morland <ijmorlan at cs.uwaterloo.ca> wrote:
> On Sun, 20 Apr 2008, Steven Bethard wrote:
>
>
> > On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
> >
> > > I have submitted a patch for review here:
> http://bugs.python.org/issue2663
> > >
> > >  glob-style patterns or a callable (for complex cases) can be provided
> > >  to filter out files or directories.
> > >
> >
> > I'm not a big fan of the sequence-or-callable argument. Why not just
> > make it a callable argument, and supply a utility function so that you
> > can write something like::
> >
> >   exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
> >   shutil.copytree(src_dir, dst_dir, exclude=exclude_func)
> >
>
>  Even if a glob pattern filter is considered useful enough to be worth
> special-casing, the glob capability should also be exposed via something
> like your excluding_patterns constructor and additionally as a function that
> can be called by another function intended for use as a callable argument.
>
>  If it is not, then doing something like "files matching these glob patterns
> except for those matching this non-glob-expressible condition and also those
> files matching this second non-glob-expressible condition" becomes painful
> because the glob part essentially needs to be re-implemented.
>
>  Isaac Morland                   CSCF Web Guru
>  DC 2554C, x36650                WWW Software Specialist



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ocean at m2.ccsnet.ne.jp  Mon Apr 21 09:54:12 2008
From: ocean at m2.ccsnet.ne.jp (ocean)
Date: Mon, 21 Apr 2008 16:54:12 +0900
Subject: [Python-Dev] Is Py_WIN_WIDE_FILENAMES still alive?
Message-ID: <000901c8a384$e3965570$0300a8c0@whiterabc2znlh>

Hello. I noticed when I removes following line in trunk/PC/pyconfig.h

#define Py_WIN_WIDE_FILENAMES

_fileio.c and posixmodule.c (and maybe more) cannot be compiled on Windows.

When Py_WIN_WIDE_FILENAMES is not defined, how should python behave?

  - call posix functions like open(2) 

  - call ANSI Win32 APIs like MoveFileA

Or maybe this macro is not used anymore?

Thank you.


From jcea at argo.es  Mon Apr 21 17:00:11 2008
From: jcea at argo.es (Jesus Cea)
Date: Mon, 21 Apr 2008 17:00:11 +0200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <47CF0B7C.109@v.loewis.de>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>
	<47CF0B7C.109@v.loewis.de>
Message-ID: <480CABFB.2080302@argo.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
| I think it would be helpful if you could analyze the crashes that bsddb
| caused on Windows. Just go back a few revisions in the subversion tree
| to reproduce the crashes.

I have no MS Windows machines in my environment :-(

| This problem has now been worked-around, by reformulating the test cases
| so that the situation doesn't occur anymore, but IMO, it should not be
| possible for an extension module to cause an interpreter abort.

Agreed. But I can't test Windows builds myself. Could anybody help me in
this issue?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
~                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSAyr9Zlgi5GaxT1NAQKzHgP/YKQ/9eLlwQfEKxePb5VtapkbfDri5T6C
dulFrvuEEQqsefBopC1K70Tm4XGNmmLpf6U4Ew5ran0dQwjRPfQH+AWo8Cloh9IC
ta8KjxHzIdl6myzitDwH1YKkDbrqdd1M5qs2QDKDjVx5c53ePHQNfLS9oOqtoWcc
XNg6ro8K4os=
=eQC0
-----END PGP SIGNATURE-----

From wolever at cs.toronto.edu  Mon Apr 21 17:56:54 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Mon, 21 Apr 2008 11:56:54 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
Message-ID: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>

Is there some sort of text encoding detection module is the standard  
library?
And, if not, is there any reason not to add one?

After some googling, I've come across this:
http://mail.python.org/pipermail/python-3000/2006-September/003537.html
But I can't find any changes that resulted from that thread.

From alexander.belopolsky at gmail.com  Mon Apr 21 18:37:57 2008
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 21 Apr 2008 16:37:57 +0000 (UTC)
Subject: [Python-Dev] unscriptable?
References: <fb6fbf560804191431h720b79a6i79caf791ed45f08a@mail.gmail.com>
Message-ID: <loom.20080421T163430-645@post.gmane.org>

> ruby: undefined method `[]=' for 1:Fixnum (NoMethodError)

I think it will be natural to unify [] error message with
the other binary ops:

Now:
>>> 1+""
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Proposal:
>>> 1[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for []: 'int' and 'int'




From skip at pobox.com  Mon Apr 21 18:44:09 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 21 Apr 2008 11:44:09 -0500
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
Message-ID: <18444.50265.618909.751390@montanaro-dyndns-org.local>


    David> Is there some sort of text encoding detection module is the
    David> standard library?  And, if not, is there any reason not to add
    David> one?

No, there's not.  I suspect the fact that you can't correctly determine the
encoding of a chunk of text 100% of the time mitigates against it.

Skip

From fuzzyman at voidspace.org.uk  Mon Apr 21 18:50:43 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 21 Apr 2008 17:50:43 +0100
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <18444.50265.618909.751390@montanaro-dyndns-org.local>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
Message-ID: <480CC5E3.1050300@voidspace.org.uk>

skip at pobox.com wrote:
>     David> Is there some sort of text encoding detection module is the
>     David> standard library?  And, if not, is there any reason not to add
>     David> one?
>
> No, there's not.  I suspect the fact that you can't correctly determine the
> encoding of a chunk of text 100% of the time mitigates against it.
>   

The only approach I know of is a heuristic based approach. e.g.

http://www.voidspace.org.uk/python/articles/guessing_encoding.shtml

(Which was 'borrowed' from docutils in the first place.)

Michael Foord
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


From lists at cheimes.de  Mon Apr 21 18:54:47 2008
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 21 Apr 2008 18:54:47 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
Message-ID: <480CC6D7.2020404@cheimes.de>

David Wolever schrieb:
> Is there some sort of text encoding detection module is the standard  
> library?
> And, if not, is there any reason not to add one?

You cannot detect the encoding unless it's explicitly defined through a
header (e.g. the UTF BOM). It's technically impossible. The best you can
do is an educated guess.

Christian

From exarkun at divmod.com  Mon Apr 21 18:59:49 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 21 Apr 2008 12:59:49 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480CC5E3.1050300@voidspace.org.uk>
Message-ID: <20080421165949.6859.924459832.divmod.quotient.33138@ohm>

On Mon, 21 Apr 2008 17:50:43 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>skip at pobox.com wrote:
>>     David> Is there some sort of text encoding detection module is the
>>     David> standard library?  And, if not, is there any reason not to add
>>     David> one?
>>
>> No, there's not.  I suspect the fact that you can't correctly determine the
>> encoding of a chunk of text 100% of the time mitigates against it.
>>
>
>The only approach I know of is a heuristic based approach. e.g.
>
>http://www.voidspace.org.uk/python/articles/guessing_encoding.shtml
>
>(Which was 'borrowed' from docutils in the first place.)

This isn't the only approach, although you're right that in general you
have to rely on heuristics.  See the charset detection features of ICU:

  http://www.icu-project.org/userguide/charsetDetection.html

I think OSAF's pyicu exposes these APIs:

  http://pyicu.osafoundation.org/

Jean-Paul

From wolever at cs.toronto.edu  Mon Apr 21 19:14:07 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Mon, 21 Apr 2008 13:14:07 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <18444.50265.618909.751390@montanaro-dyndns-org.local>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
Message-ID: <91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>

On 21-Apr-08, at 12:44 PM, skip at pobox.com wrote:
>
>     David> Is there some sort of text encoding detection module is the
>     David> standard library?  And, if not, is there any reason not  
> to add
>     David> one?
> No, there's not.  I suspect the fact that you can't correctly  
> determine the
> encoding of a chunk of text 100% of the time mitigates against it.
Sorry, I wasn't very clear what I was asking.

I was thinking about making an educated guess -- just like chardet  
(http://chardet.feedparser.org/).

This is useful when you get a hunk of data which _should_ be some  
sort of intelligible text from the Big Scary Internet (say, a posted  
web form or email message), and you want to do something useful with  
it (say, search the content).

From g.brandl at gmx.net  Mon Apr 21 19:27:27 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 21 Apr 2008 19:27:27 +0200
Subject: [Python-Dev] unscriptable?
In-Reply-To: <loom.20080421T163430-645@post.gmane.org>
References: <fb6fbf560804191431h720b79a6i79caf791ed45f08a@mail.gmail.com>
	<loom.20080421T163430-645@post.gmane.org>
Message-ID: <fuiijq$igj$1@ger.gmane.org>

Alexander Belopolsky schrieb:
>> ruby: undefined method `[]=' for 1:Fixnum (NoMethodError)
> 
> I think it will be natural to unify [] error message with
> the other binary ops:
> 
> Now:
>>>> 1+""
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for +: 'int' and 'str'
> 
> Proposal:
>>>> 1[2]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for []: 'int' and 'int'

This is misleading, since it suggests that you can remedy this
by using another type in the subscript, which isn't possible.

Georg


From guido at python.org  Mon Apr 21 19:29:38 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 21 Apr 2008 10:29:38 -0700
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
Message-ID: <ca471dc20804211029r4e58bb86odcca8ae8d44ba894@mail.gmail.com>

On Sun, Apr 20, 2008 at 5:25 PM, Steven Bethard
<steven.bethard at gmail.com> wrote:
>
> On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>  > I have submitted a patch for review here: http://bugs.python.org/issue2663
>  >
>  >  glob-style patterns or a callable (for complex cases) can be provided
>  >  to filter out files or directories.
>
>  I'm not a big fan of the sequence-or-callable argument. Why not just
>  make it a callable argument, and supply a utility function so that you
>  can write something like::
>
>     exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
>     shutil.copytree(src_dir, dst_dir, exclude=exclude_func)
>
>  ?

Agreed. Type testing is fraught with problems.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Mon Apr 21 19:28:48 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 21 Apr 2008 19:28:48 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480CC6D7.2020404@cheimes.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<480CC6D7.2020404@cheimes.de>
Message-ID: <fuiimb$igj$2@ger.gmane.org>

Christian Heimes schrieb:
> David Wolever schrieb:
>> Is there some sort of text encoding detection module is the standard  
>> library?
>> And, if not, is there any reason not to add one?
> 
> You cannot detect the encoding unless it's explicitly defined through a
> header (e.g. the UTF BOM). It's technically impossible. The best you can
> do is an educated guess.

Exactly, and in light of that, I'm -1 for such a standard module.
We've enough issues with modules implementing (apparently) fully
specified standards. :)

Georg


From guido at python.org  Mon Apr 21 19:38:03 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 21 Apr 2008 10:38:03 -0700
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <fuiimb$igj$2@ger.gmane.org>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<480CC6D7.2020404@cheimes.de> <fuiimb$igj$2@ger.gmane.org>
Message-ID: <ca471dc20804211038n667fdaf5rac62b0a73eebf0ce@mail.gmail.com>

To the contrary, an encoding-guessing module is often needed, and
guessing can be done with a pretty high success rate. Other Unicode
libraries (e.g. ICU) contain guessing modules. I suppose the API could
return two values: the guessed encoding and a confidence indicator.
Note that the locale settings might figure in the guess.

On Mon, Apr 21, 2008 at 10:28 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Christian Heimes schrieb:
>
> > David Wolever schrieb:
>  >> Is there some sort of text encoding detection module is the standard
>  >> library?
>  >> And, if not, is there any reason not to add one?
>  >
>  > You cannot detect the encoding unless it's explicitly defined through a
>  > header (e.g. the UTF BOM). It's technically impossible. The best you can
>  > do is an educated guess.
>
>  Exactly, and in light of that, I'm -1 for such a standard module.
>  We've enough issues with modules implementing (apparently) fully
>  specified standards. :)
>
>  Georg
>
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Apr 21 19:00:54 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 21 Apr 2008 12:00:54 -0500
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480CC5E3.1050300@voidspace.org.uk>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<480CC5E3.1050300@voidspace.org.uk>
Message-ID: <18444.51270.116350.591529@montanaro-dyndns-org.local>


    Michael> The only approach I know of is a heuristic based approach. e.g.

    Michael> http://www.voidspace.org.uk/python/articles/guessing_encoding.shtml

    Michael> (Which was 'borrowed' from docutils in the first place.)

Yes, I implemented a heuristic approach for the Musi-Cal web server.  I was
able to rely on domain knowledge to guess correctly almost all the time.
The heuristic was that almost all form submissions came from the US and the
rest which didn't came from Western Europe.  Python could never embed such a
narrow-focused heuristic into its core distribution.

Skip


From skip at pobox.com  Mon Apr 21 19:56:27 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 21 Apr 2008 12:56:27 -0500
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <ca471dc20804211038n667fdaf5rac62b0a73eebf0ce@mail.gmail.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<480CC6D7.2020404@cheimes.de> <fuiimb$igj$2@ger.gmane.org>
	<ca471dc20804211038n667fdaf5rac62b0a73eebf0ce@mail.gmail.com>
Message-ID: <18444.54603.483042.133611@montanaro-dyndns-org.local>


    Guido> Note that the locale settings might figure in the guess.

Alas, locale settings in a web server have little or nothing to do with the
locale settings in the client submitting the form.

Skip

From greg at krypto.org  Mon Apr 21 20:10:24 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 21 Apr 2008 11:10:24 -0700
Subject: [Python-Dev] very bad network performance
In-Reply-To: <d2155e360804141641n543ebc8bp77a16c1289b92e45@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
	<932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
	<ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
	<d2155e360804141641n543ebc8bp77a16c1289b92e45@mail.gmail.com>
Message-ID: <52dc1c820804211110g5827ba59t60fcde7f0c7b571c@mail.gmail.com>

On Mon, Apr 14, 2008 at 4:41 PM, Curt Hagenlocher <curt at hagenlocher.org>
wrote:

> On Mon, Apr 14, 2008 at 4:19 PM, Guido van Rossum <guido at python.org>
> wrote:
> >
> > But why was imaplib apparently specifying 10MB? Did it know there was
> > that much data? Or did it just not want to bother looping over all the
> > data in smaller buffer increments (e.g. 64K, which is probably the max
> > of what most TCP stacks will give you)?
>
> I'm going to guess that the code in question is
>
>                size = int(self.mo.group('size'))
>                if __debug__:
>                    if self.debug >= 4:
>                        self._mesg('read literal size %s' % size)
>                data = self.read(size)
>
> It's reading however many bytes are reported by the server as the size.
>
> > If I'm right with my hunch that the TCP stack will probably clamp at
> > 64K, perhaps we should use min(system limit, max(requested size,
> > buffer size))?
>
> I have indeed missed the point of the read buffer size.  This would work.
>

The 64K hunch is wrong.  The system limit can be found using
getsockopt(...SO_RCVBUF...).  It can easily be (and often is) set to many
megabytes either at a system default level or on a per socket level by the
user using setsockopt.  When the system default is that large, limiting by
the system limit would not help the 10mb read case.

Even smaller allocations like 64K cause problems as mentioned in issue
1092502 linking to this twisted
http://twistedmatrix.com/trac/ticket/1079bug.  twisted's solution was
to make the string object returned by a recv as
short lived as possible by copying it into a StringIO.  We could do the same
in _fileobject.read() and readline().

I have attached a patch to issue 2632 that changes socket to use StringIO
for its read buffer and keeps the lifetime of strings returned by recv() as
short as possible when appropriate.  It also refuses to call recv() with a
size smaller than default_bufsize within read() [the source of the
performance problem].  That changes internal recv() call behavior over the
existing code after the issue 1092502 "fix" was applied to use min() rather
than max(), but it is -not- a significant change over the pre-1092502 "fix"
behavior that exists in all released versions of python (it already chose
the larger of two values for recv sizes).

The main difference behind the scenes?  StringIO is using realloc only to
increase its size while recv() was using realloc to shrink the allocation
size and many of these recv()ed shrunken strings were being held onto in a
list before the final value was constructed.

I suggest continuing the discussion within issue 2632 to keep better track
of it.

My socket-strio patch in 2632 needs more testing (it passed socket, http*
and url* tests) and verification that both issue's problems are indeed gone
but they should be.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080421/6f8c4938/attachment.htm 

From tonynelson at georgeanelson.com  Mon Apr 21 20:34:50 2008
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Mon, 21 Apr 2008 14:34:50 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
Message-ID: <p04330101c4328e0bb684@[192.168.123.162]>

At 1:14 PM -0400 4/21/08, David Wolever wrote:
>On 21-Apr-08, at 12:44 PM, skip at pobox.com wrote:
>>
>>     David> Is there some sort of text encoding detection module is the
>>     David> standard library?  And, if not, is there any reason not
>> to add
>>     David> one?
>> No, there's not.  I suspect the fact that you can't correctly
>> determine the
>> encoding of a chunk of text 100% of the time mitigates against it.
>Sorry, I wasn't very clear what I was asking.
>
>I was thinking about making an educated guess -- just like chardet
>(http://chardet.feedparser.org/).
>
>This is useful when you get a hunk of data which _should_ be some
>sort of intelligible text from the Big Scary Internet (say, a posted
>web form or email message), and you want to do something useful with
>it (say, search the content).

Feedparser.org's chardet can't guess 'latin1', so it should be used as a
last resort, just as the docs say.
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From martin at v.loewis.de  Mon Apr 21 23:29:18 2008
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 21 Apr 2008 23:29:18 +0200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480CABFB.2080302@argo.es>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
Message-ID: <480D072E.8080100@v.loewis.de>

> | This problem has now been worked-around, by reformulating the test cases
> | so that the situation doesn't occur anymore, but IMO, it should not be
> | possible for an extension module to cause an interpreter abort.
> 
> Agreed. But I can't test Windows builds myself. Could anybody help me in
> this issue?.

Sure. If you post a patch, I'm sure someone will be able to test it on
Windows and report results; when it's committed, also the buildbots will
test it.

Regards,
Martin


From martin at v.loewis.de  Mon Apr 21 23:31:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 21 Apr 2008 23:31:06 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
Message-ID: <480D079A.2020904@v.loewis.de>

> This is useful when you get a hunk of data which _should_ be some  
> sort of intelligible text from the Big Scary Internet (say, a posted  
> web form or email message), and you want to do something useful with  
> it (say, search the content).

I don't think that should be part of the standard library. People
will mistake what it tells them for certain.

Regards,
Martin

From rbp at isnomore.net  Mon Apr 21 23:37:20 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Mon, 21 Apr 2008 18:37:20 -0300
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480D079A.2020904@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
Message-ID: <20080421213720.GB16282@isnomore.net>

On Mon, Apr 21 2008 at 06:31:06PM BRT, "\"Martin v. L?wis\"" <martin at v.loewis.de> wrote:
> > This is useful when you get a hunk of data which _should_ be some  
> > sort of intelligible text from the Big Scary Internet (say, a posted  
> > web form or email message), and you want to do something useful with  
> > it (say, search the content).
> 
> I don't think that should be part of the standard library. People
> will mistake what it tells them for certain.

Maybe call it "charguess", then?


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From phd at phd.pp.ru  Tue Apr 22 00:17:01 2008
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Tue, 22 Apr 2008 02:17:01 +0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <20080421213720.GB16282@isnomore.net>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<20080421213720.GB16282@isnomore.net>
Message-ID: <20080421221701.GA13372@phd.pp.ru>

On Mon, Apr 21, 2008 at 06:37:20PM -0300, Rodrigo Bernardo Pimentel wrote:
> On Mon, Apr 21 2008 at 06:31:06PM BRT, "\"Martin v. L??wis\"" <martin at v.loewis.de> wrote:
> > > This is useful when you get a hunk of data which _should_ be some  
> > > sort of intelligible text from the Big Scary Internet (say, a posted  
> > > web form or email message), and you want to do something useful with  
> > > it (say, search the content).
> > 
> > I don't think that should be part of the standard library. People
> > will mistake what it tells them for certain.
> 
> Maybe call it "charguess", then?

   The famous chardet returns probablity of its guessing:

>>> import chardet
>>> chardet.detect("dabc")
{'confidence': 1.0, 'encoding': 'ascii'}
>>> chardet.detect("????")
{'confidence': 0.98999999999999999, 'encoding': 'KOI8-R'}

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From greg.ewing at canterbury.ac.nz  Tue Apr 22 02:26:49 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 22 Apr 2008 12:26:49 +1200
Subject: [Python-Dev] thoughts on having EOFError inherit from
 EnvironmentError?
In-Reply-To: <20080419121255.150520e9.steve@pearwood.info>
References: <52dc1c820804121501k5c3d0190k9e62cb4c47878d58@mail.gmail.com>
	<ca471dc20804141212l3bdcd526o5ae42f41119ec575@mail.gmail.com>
	<48040C08.504@canterbury.ac.nz>
	<ca471dc20804141911i47f6a0bbq9d4445be892f80f4@mail.gmail.com>
	<48052BC1.40202@canterbury.ac.nz>
	<ca471dc20804151536i936a25cs88e1f5794b8092df@mail.gmail.com>
	<48057537.3060003@canterbury.ac.nz>
	<ca471dc20804152139j3484f4a3q1781707b5a66bd63@mail.gmail.com>
	<4806997F.50000@canterbury.ac.nz>
	<loom.20080418T091149-177@post.gmane.org>
	<48093A67.5070702@canterbury.ac.nz>
	<20080419121255.150520e9.steve@pearwood.info>
Message-ID: <480D30C9.2050805@canterbury.ac.nz>

Steven wrote:
> It might help if you explain what sort of actual things that the user does 
> wrong that you are talking about.

Usually it's some kind of parsing error. Or it might be a
network connection getting closed unexpectedly.

Essentially it's anything due to factors outside the
program's control, but which aren't detected and reported
by any of the built-in IOError or OSError exceptions.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Apr 22 02:50:44 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 22 Apr 2008 12:50:44 +1200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
Message-ID: <480D3664.5080001@canterbury.ac.nz>

Steven Bethard wrote:
> I'm not a big fan of the sequence-or-callable argument. Why not just
> make it a callable argument, and supply a utility function

Or have two different keyword arguments, one for a sequence
and one for a callable.

-- 
Greg

From wolever at cs.toronto.edu  Tue Apr 22 03:41:51 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Mon, 21 Apr 2008 21:41:51 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480D079A.2020904@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
Message-ID: <91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>

On 21-Apr-08, at 5:31 PM, Martin v. L?wis wrote:
>> This is useful when you get a hunk of data which _should_ be some
>> sort of intelligible text from the Big Scary Internet (say, a posted
>> web form or email message), and you want to do something useful with
>> it (say, search the content).
> I don't think that should be part of the standard library. People
> will mistake what it tells them for certain.
As Oleg mentioned, if the method is called something like  
'guess_encoding', I think we could live with clear consciences.

IMO, encoding estimation is something that many web programs will  
have to deal with, so it might as well be built in; I would prefer  
the option to run `text=input.encode('guess')` (or something similar)  
than relying on an external dependency or worse yet using a hand- 
rolled algorithm.

From jimjjewett at gmail.com  Tue Apr 22 05:30:18 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 21 Apr 2008 23:30:18 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
Message-ID: <fb6fbf560804212030l3cc41ed8gbadc151ee3aadd36@mail.gmail.com>

David Wolever wrote:

> IMO, encoding estimation is something that
> many web programs will have to deal with,
> so it might as well be built in; I would prefer
> the option to run `text=input.encode('guess')`
> (or something similar) than relying on an external
> dependency or worse yet using a hand-rolled
> algorithm

The (still draft) html5 spec is trying to get error-correction
standardized, so it includes all sort of "if this fails, do X".
Encoding detection will be standardized, so there will be an external
standard that we can reference.

http://dev.w3.org/html5/spec/Overview.html#determining

Note that this portion of the spec is probably not stable yet, as
there was some new analysis on which "wrong" answers provided better
results on real world web pages.

e.g.,

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2008-March/014127.html

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2008-March/014190.html

There was also a recent analysis of how many characters it takes to
sniff successfully X% of the time on today's web, though I'm not
finding it at the moment.

-jJ

From martin at v.loewis.de  Tue Apr 22 06:30:33 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Apr 2008 06:30:33 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
Message-ID: <480D69E9.7020004@v.loewis.de>

> IMO, encoding estimation is something that many web programs will have
> to deal with

Can you please explain why that is? Web programs should not normally
have the need to detect the encoding; instead, it should be specified
always - unless you are talking about browsers specifically, which
need to support web pages that specify the encoding incorrectly.

> so it might as well be built in; I would prefer the option
> to run `text=input.encode('guess')` (or something similar) than relying
> on an external dependency or worse yet using a hand-rolled algorithm.

Ok, let me try differently then. Please feel free to post a patch to
bugs.python.org, and let other people rip it apart.

For example, I don't think it should be a codec, as I can't imagine it
working on streams.

Regards,
Martin

From nnorwitz at gmail.com  Tue Apr 22 06:46:10 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 21 Apr 2008 21:46:10 -0700
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <fu1s2n$i84$1@ger.gmane.org>
References: <20080414205105.B45C11E4005@bag.python.org>
	<4803C735.10100@cheimes.de>
	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>
	<4803D319.5010000@cheimes.de>
	<ee2a432c0804142206j65978c2dse4a9645d9dd2d351@mail.gmail.com>
	<fu1s2n$i84$1@ger.gmane.org>
Message-ID: <ee2a432c0804212146w87421bam3d522b1d7f141c75@mail.gmail.com>

On Tue, Apr 15, 2008 at 2:21 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Neal Norwitz wrote:
>  > Iteration with the dict methods (e.g., keys -> iterkeys()),
>  > map/zip/filter returning iterator rather than list.
>
>  That's only an optimisation, it's not functionally required. A list behaves
>  like an iterator in most use cases, so it's rather unlikely that Py3 code will
>  break in the backport because of this (and it's very unlikely that static
>  analysis can catch the cases where it breaks). There should be a rule to
>  optimise "list(map(...))" into "map(...)" and "list(x.keys())" into plain
>  "x.keys()" etc., but I don't think there is much more to do here.

It's not just an optimization if a copy won't fit in memory.  I'd like
the solution to be closer to 100% than 95%.

>  > int -> (int, long)
>
>  Is there any case where this might be required? I don't see any reason why
>  back-converted Py3 code should break here. What would "long()" be needed for
>  in working Py3 code that "int()" doesn't provide in Py2?
>
>  Although you might have been referring to "isinstance(x, int)" in Py3?

Yes, sorry, I wasn't explicit.  isinstance is specifically what I was
referring to (or other type checks such as type(x) in (int, long)).

>  > str -> basestring or (str, unicode)
>
>  This is an issue, although I think that Py3 is explicit enough here to make
>  this easy to handle by static renaming (and maybe optimising "isinstance(s,
>  (str, bytes))" into "..., basestring))").
>
>
>
>  > __bool__ -> __nonzero__
>  > exec/execfile
>  > input -> rawinput
>
>  Also valid issues that can be handled through renaming and static syntactic
>  adjustments.
>
>
>
>  > Most things that have a fixer in 2to3 would also require one in 3to2.
>
>  I think the more explicit syntax in Py3 will actually make it easier to
>  back-convert the code statically from 3to2 than to migrate it from 2to3.

Sure, that's the idea.

I haven't seen any action on 3to2 (although I'm very behind on email).
 Stefan, could you try to implement some of these and report back how
it works?

n

From schmir at gmail.com  Tue Apr 22 08:44:48 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 22 Apr 2008 08:44:48 +0200
Subject: [Python-Dev] very bad network performance
In-Reply-To: <52dc1c820804211110g5827ba59t60fcde7f0c7b571c@mail.gmail.com>
References: <932f8baf0804140912u54adc7d5md7261541857f21bd@mail.gmail.com>
	<d2155e360804141110i7b8d0f8dm9bafe6881b804860@mail.gmail.com>
	<932f8baf0804141429v666672a0j8833d8bcd3cd44d5@mail.gmail.com>
	<d2155e360804141519k2ed5afb8h379a346773e8eca1@mail.gmail.com>
	<932f8baf0804141557u5d1a06e6q7112dd4d606e079c@mail.gmail.com>
	<ca471dc20804141619u7dcb8af7u19ccef032ec97ee0@mail.gmail.com>
	<d2155e360804141641n543ebc8bp77a16c1289b92e45@mail.gmail.com>
	<52dc1c820804211110g5827ba59t60fcde7f0c7b571c@mail.gmail.com>
Message-ID: <932f8baf0804212344w3722c1eewe954b59498e708c9@mail.gmail.com>

On Mon, Apr 21, 2008 at 8:10 PM, Gregory P. Smith <greg at krypto.org> wrote:

>
>
>
> The 64K hunch is wrong.  The system limit can be found using
> getsockopt(...SO_RCVBUF...).  It can easily be (and often is) set to many
> megabytes either at a system default level or on a per socket level by the
> user using setsockopt.  When the system default is that large, limiting by
> the system limit would not help the 10mb read case.
>

but it would help in the 100mb read case.


>
> Even smaller allocations like 64K cause problems as mentioned in issue
> 1092502 linking to this twisted http://twistedmatrix.com/trac/ticket/1079bug.  twisted's solution was to make the string object returned by a recv as
> short lived as possible by copying it into a StringIO.  We could do the same
> in _fileobject.read() and readline().
>

this approach look reasonable to me.

- Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080422/990b817b/attachment.htm>

From ziade.tarek at gmail.com  Tue Apr 22 09:56:47 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Apr 2008 09:56:47 +0200
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
Message-ID: <94bdd2610804220056w24a34d07q16b1151796f1b9d7@mail.gmail.com>

On Mon, Apr 21, 2008 at 2:25 AM, Steven Bethard
<steven.bethard at gmail.com> wrote:
>
> On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>  > I have submitted a patch for review here: http://bugs.python.org/issue2663
>  >
>  >  glob-style patterns or a callable (for complex cases) can be provided
>  >  to filter out files or directories.
>
>  I'm not a big fan of the sequence-or-callable argument. Why not just
>  make it a callable argument, and supply a utility function so that you
>  can write something like::
>
>     exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
>     shutil.copytree(src_dir, dst_dir, exclude=exclude_func)
>
>  ?

I made another draft based on a single callable argument to try out:
http://bugs.python.org/file10073/shutil.copytree.filtering.patch

The callable takes the src directory + its content as a list, and
returns filter eligible for exclusion

That makes me wonder, like Alexander said on the bug tracker:
In the glob-style patterns callable, do we want to deal with absolute paths ?


Tarek

From stefan_ml at behnel.de  Tue Apr 22 10:45:49 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 22 Apr 2008 10:45:49 +0200
Subject: [Python-Dev] r62342 - python/branches/py3k/Objects/bytesobject.c
In-Reply-To: <ee2a432c0804212146w87421bam3d522b1d7f141c75@mail.gmail.com>
References: <20080414205105.B45C11E4005@bag.python.org>	<4803C735.10100@cheimes.de>	<43aa6ff70804141418w414cbe0bl8eb82ea77e8559a1@mail.gmail.com>	<4803D319.5010000@cheimes.de>	<ee2a432c0804142206j65978c2dse4a9645d9dd2d351@mail.gmail.com>	<fu1s2n$i84$1@ger.gmane.org>
	<ee2a432c0804212146w87421bam3d522b1d7f141c75@mail.gmail.com>
Message-ID: <480DA5BD.9080300@behnel.de>

Neal Norwitz wrote:
> I haven't seen any action on 3to2 (although I'm very behind on email).
>  Stefan, could you try to implement some of these and report back how
> it works?

No, sorry, that's too low a priority for me currently.

Stefan


From mal at egenix.com  Tue Apr 22 12:31:34 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 22 Apr 2008 12:31:34 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480D079A.2020904@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
Message-ID: <480DBE86.2000507@egenix.com>

On 2008-04-21 23:31, Martin v. L?wis wrote:
>> This is useful when you get a hunk of data which _should_ be some  
>> sort of intelligible text from the Big Scary Internet (say, a posted  
>> web form or email message), and you want to do something useful with  
>> it (say, search the content).
> 
> I don't think that should be part of the standard library. People
> will mistake what it tells them for certain.

+1

I also think that it's better to educate people to add (correct)
encoding information to their text data, rather than give them a
guess mechanism...

http://chardet.feedparser.org/docs/faq.html#faq.yippie

chardet is based on the Mozilla algorithm and at least in
my experience that algorithm doesn't work too well.

The Mozilla algorithm may work for Asian encodings due to the fact
that those encodings are usually also bound to a specific language
(and you can then use character and word frequency analysis), but
for encodings which can encode far more than just a single language
(e.g. UTF-8 or Latin-1), the correct detection rate is rather low.

The problem becomes completely even more difficult when leaving
the normal text domain or when mixing languages in the same
text, e.g. when trying to detect source code with comments using
a non-ASCII encoding.

The "trick" to just pass the text through a codec and see whether
it roundtrips also doesn't necessarily help: Latin-1, for example,
will always round-trip, since Latin-1 is a subset of Unicode.

IMHO, more research has to be done into this area before a
"standard" module can be added to the Python's stdlib... and
who knows, perhaps we're lucky and by the time everyone is
using UTF-8 anyway :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 22 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ndbecker2 at gmail.com  Tue Apr 22 13:17:24 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 22 Apr 2008 07:17:24 -0400
Subject: [Python-Dev] pydoc works with eggs? (python-2.5.1)
Message-ID: <fukhg4$ke6$1@ger.gmane.org>

pydoc blew up when I tried to view doc for pytools module, which is an egg:

pydoc -p 8082
pydoc server ready at http://localhost:8082/
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52915)
Traceback (most recent call last):
  File "/usr/lib64/python2.5/SocketServer.py", line 222, in handle_request
    self.process_request(request, client_address)
  File "/usr/lib64/python2.5/SocketServer.py", line 241, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.5/SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.5/SocketServer.py", line 522, in __init__
    self.handle()
  File "/usr/lib64/python2.5/BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "/usr/lib64/python2.5/BaseHTTPServer.py", line 310, in handle_one_request
    method()
  File "/usr/lib64/python2.5/pydoc.py", line 1924, in do_GET
    self.send_document(describe(obj), html.document(obj, path))
  File "/usr/lib64/python2.5/pydoc.py", line 321, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "/usr/lib64/python2.5/pydoc.py", line 672, in docmodule
    contents.append(self.document(value, key, name, fdict, cdict))
  File "/usr/lib64/python2.5/pydoc.py", line 322, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "/usr/lib64/python2.5/pydoc.py", line 807, in docclass
    lambda t: t[1] == 'method')
  File "/usr/lib64/python2.5/pydoc.py", line 735, in spill
    funcs, classes, mdict, object))
  File "/usr/lib64/python2.5/pydoc.py", line 323, in document
    if inspect.isroutine(object): return self.docroutine(*args)
  File "/usr/lib64/python2.5/pydoc.py", line 891, in docroutine
    getdoc(object), self.preformat, funcs, classes, methods)
  File "/usr/lib64/python2.5/pydoc.py", line 79, in getdoc
    result = inspect.getdoc(object) or inspect.getcomments(object)
  File "/usr/lib64/python2.5/inspect.py", line 521, in getcomments
    lines, lnum = findsource(object)
  File "/usr/lib64/python2.5/inspect.py", line 510, in findsource
    if pat.match(lines[lnum]): break
IndexError: list index out of range
----------------------------------------



From janssen at parc.com  Tue Apr 22 17:14:43 2008
From: janssen at parc.com (Bill Janssen)
Date: Tue, 22 Apr 2008 08:14:43 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480DBE86.2000507@egenix.com> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de> <480DBE86.2000507@egenix.com>
Message-ID: <08Apr22.081453pdt."58696"@synergy1.parc.xerox.com>

> IMHO, more research has to be done into this area before a
> "standard" module can be added to the Python's stdlib... and
> who knows, perhaps we're lucky and by the time everyone is
> using UTF-8 anyway :-)

I walked over to our computational linguistics group and asked.  This
is often combined with language guessing (which uses a similar
approach, but using characters instead of bytes), and apparently can
usually be done with high confidence.  Of course, they're usually
looking at clean texts, not random "stuff".  I'll see if I can get
some references and report back -- most of the research on this was
done in the 90's.

Bill

From ronaldoussoren at mac.com  Tue Apr 22 17:47:53 2008
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 22 Apr 2008 17:47:53 +0200
Subject: [Python-Dev] configure error: "rm: conftest.dSYM: is a
	directory"
In-Reply-To: <18423.53311.150471.667891@montanaro-dyndns-org.local>
References: <18423.53311.150471.667891@montanaro-dyndns-org.local>
Message-ID: <403F765C-2EFF-4B9B-9780-835EDDC54F43@mac.com>


On 5 Apr, 2008, at 21:17, skip at pobox.com wrote:
> I just noticed this error message during configure:
>
>    checking whether gcc accepts -Olimit 1500... no
>    checking whether gcc supports ParseTuple __format__... no
>    checking whether pthreads are available without options... yes
>    checking whether g++ also accepts flags for thread support... no
>    checking for ANSI C header files... rm: conftest.dSYM: is a  
> directory
>    rm: conftest.dSYM: is a directory
>    yes
>    checking for sys/types.h... yes
>    checking for sys/stat.h... yes
>    checking for stdlib.h... yes
>    checking for string.h... yes
>
> Note the "rm: conftest.dSYM: is a directory".  This occurred a few  
> times
> during the configure process.  Didn't cause it to conk out, but is  
> annoying.

I've looked into this issue. It is harmless and caused by an  
interaction between
AC_TRY_RUN and gcc on leopard.

Gcc generates '.dSYM' directories when linking with debugging enabled.  
These
directories contain detached debugging information (see man dsymutil).

AC_TRY_RUN tries to remove 'conftest.*' using rm, without the -r flag.  
The end result
is an error message during configure and a 'config.dSYM' turd.

AFAIK this not easily fixed without changing the definition of  
AC_TRY_RUN, at least not
without crude hacks.

Ronald
>
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080422/3180335c/attachment.bin>

From wolever at cs.toronto.edu  Tue Apr 22 17:48:07 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Tue, 22 Apr 2008 11:48:07 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480D69E9.7020004@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
Message-ID: <091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>

On 22-Apr-08, at 12:30 AM, Martin v. L?wis wrote:
>> IMO, encoding estimation is something that many web programs will  
>> have
>> to deal with
> Can you please explain why that is? Web programs should not normally
> have the need to detect the encoding; instead, it should be specified
> always - unless you are talking about browsers specifically, which
> need to support web pages that specify the encoding incorrectly.
Two cases come immediately to mind: email and web forms.
When a web browser POSTs data, there is no standard way of  
communicating which encoding it's using.  There are some hints which  
make it easier (accept-charset attributes, the encoding used to send  
the page to the browser), but no guarantees.
Email is a smaller problem, because it usually has a helpful content- 
type header, but that's no guarantee.

Now, at the moment, the only data I have to support this claim is my  
experience with DrProject in non-English locations.
If I'm the only one who has had these sorts of problems, I'll go back  
to "Unicode for Dummies".

>> so it might as well be built in; I would prefer the option
>> to run `text=input.encode('guess')` (or something similar) than  
>> relying
>> on an external dependency or worse yet using a hand-rolled algorithm.
> Ok, let me try differently then. Please feel free to post a patch to
> bugs.python.org, and let other people rip it apart.
> For example, I don't think it should be a codec, as I can't imagine it
> working on streams.

As things frequently are, it seems like this is a much larger problem  
that I originally believed.

I'll go back and take another look at the problem, then come back if  
new revelations appear.

From janssen at parc.com  Tue Apr 22 18:33:18 2008
From: janssen at parc.com (Bill Janssen)
Date: Tue, 22 Apr 2008 09:33:18 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <08Apr22.081453pdt."58696"@synergy1.parc.xerox.com> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de> <480DBE86.2000507@egenix.com>
	<08Apr22.081453pdt."58696"@synergy1.parc.xerox.com>
Message-ID: <08Apr22.093328pdt."58696"@synergy1.parc.xerox.com>

The 2002 paper "A language and character set determination method
based on N-gram statistics" by Izumi Suzuki and Yoshiki Mikami and
Ario Ohsato and Yoshihide Chubachi seems to me a pretty good way to go
about this.  They're looking at "LSE"s, language-script-encoding
triples; a "script" is a way of using a particular character set to
write in a particular language.

Their system has these requirements:

R1. the response must be either "correct answer" or "unable to detect"
    where "unable to detect" includes "other than registered" [the
    registered set of LSEs];

R2. Applicable to multi-LSE texts;

R3. never accept a wrong answer, even when the program does not have
    enough data on an LSE; and

R4. applicable to any LSE text.

So, no wrong answers.

The biggest disadvantage would seem to be that the registration data
for a particular LSE is kind of bulky; on the order of 10,000
shift-codons, each of three bytes, about 30K uncompressed.

http://portal.acm.org/ft_gateway.cfm?id=772759&type=pdf

Bill

> > IMHO, more research has to be done into this area before a
> > "standard" module can be added to the Python's stdlib... and
> > who knows, perhaps we're lucky and by the time everyone is
> > using UTF-8 anyway :-)
> 
> I walked over to our computational linguistics group and asked.  This
> is often combined with language guessing (which uses a similar
> approach, but using characters instead of bytes), and apparently can
> usually be done with high confidence.  Of course, they're usually
> looking at clean texts, not random "stuff".  I'll see if I can get
> some references and report back -- most of the research on this was
> done in the 90's.
> 
> Bill

From steven.bethard at gmail.com  Tue Apr 22 19:04:45 2008
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 22 Apr 2008 11:04:45 -0600
Subject: [Python-Dev] A smarter shutil.copytree ?
In-Reply-To: <94bdd2610804220056w24a34d07q16b1151796f1b9d7@mail.gmail.com>
References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com>
	<ca471dc20804171006r6a825a8fj7a954a5c8ef6b139@mail.gmail.com>
	<94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com>
	<94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com>
	<d11dcfba0804201725o9dfbdc6g6c19cc6ccb33ba04@mail.gmail.com>
	<94bdd2610804220056w24a34d07q16b1151796f1b9d7@mail.gmail.com>
Message-ID: <d11dcfba0804221004q4af5a49cw8e33dafa2579d3fd@mail.gmail.com>

On Tue, Apr 22, 2008 at 1:56 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Mon, Apr 21, 2008 at 2:25 AM, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On Sun, Apr 20, 2008 at 4:15 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>  >  > I have submitted a patch for review here: http://bugs.python.org/issue2663
>  >  >
>  >  >  glob-style patterns or a callable (for complex cases) can be provided
>  >  >  to filter out files or directories.
>  >
>  >  I'm not a big fan of the sequence-or-callable argument. Why not just
>  >  make it a callable argument, and supply a utility function so that you
>  >  can write something like::
>  >
>  >     exclude_func = shutil.excluding_patterns('*.tmp', 'test_dir2')
>  >     shutil.copytree(src_dir, dst_dir, exclude=exclude_func)
>
>  I made another draft based on a single callable argument to try out:
>  http://bugs.python.org/file10073/shutil.copytree.filtering.patch
>
>  The callable takes the src directory + its content as a list, and
>  returns filter eligible for exclusion

FWIW, that looks better to me.

>  That makes me wonder, like Alexander said on the bug tracker:
>  In the glob-style patterns callable, do we want to deal with absolute paths ?

I think that it would be okay to document that
shutil.ignore_patterns() only accepts patterns matching individual
filenames (not complex paths). If someone needs to do something with
absolute paths, then they can write their own 'ignore' function,
right?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
 --- Bucky Katt, Get Fuzzy

From martin at v.loewis.de  Tue Apr 22 20:06:16 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Apr 2008 20:06:16 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
Message-ID: <480E2918.7090006@v.loewis.de>

> When a web browser POSTs data, there is no standard way of communicating
> which encoding it's using.

That's just not true. Web browser should and do use the encoding of the
web page that originally contained the form.

> There are some hints which make it easier
> (accept-charset attributes, the encoding used to send the page to the
> browser), but no guarantees.

Not true. The latter is guaranteed (unless you assume bugs - but if
you do, can you present a specific browser that has that bug?)

> Email is a smaller problem, because it usually has a helpful
> content-type header, but that's no guarantee.

Then assume windows-1252. Mailers who don't use MIME for non-ASCII
characters mostly died 10 years ago; those people who continue to
use them likely can accept occasional moji-bake (or else they would
have switched long ago).

> Now, at the moment, the only data I have to support this claim is my
> experience with DrProject in non-English locations.
> If I'm the only one who has had these sorts of problems, I'll go back to
> "Unicode for Dummies".

For web forms, I always encode the pages in UTF-8, and that always
works.

For email, I once added encoding processing to the pipermail (the
mailman archiver), and that also always works.

> I'll go back and take another look at the problem, then come back if new
> revelations appear.

Good luck!

Martin

From tnelson at onresolve.com  Tue Apr 22 20:42:10 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 22 Apr 2008 11:42:10 -0700
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480CABFB.2080302@argo.es>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>
	<fpuf2u$6se$1@ger.gmane.org>
	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>
	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>
	<47C32D2D.1030206@free.fr>
	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>
	<47C4813F.2080403@v.loewis.de>
	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>
	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>
	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>
	<20080227141322.GA5871@amk-desktop.matrixgroup.net>
	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>

Hi Jesus,

> Martin v. L?wis wrote:
> | I think it would be helpful if you could analyze the crashes that
> | bsddb caused on Windows. Just go back a few revisions in the
> | subversion tree to reproduce the crashes.
>
> I have no MS Windows machines in my environment :-(

I remember those rampant BSDDB crashes on Windows well.  I brought this up with Martin at PyCon; I really don't think we can fault BSDDB here -- basically, the tests weren't cleaning up their environment in the right order, so BSDDB was getting passed completely and utterly bogus values.  I *think* I managed to persuade Martin that this was indeed our fault, and we can't really hold BSDDB accountable.  (My argument being that if a 3rd party app says the behaviour of a method is undefined if you pass it a null pointer, and you pass it a null pointer, and it crashes your program, it's your fault, not theirs.)

Once this was addressed, the BSDDB tests ran more or less on Windows 32-bit without error.  Windows x64 was another matter though -- I traced the problem down to wildly conflicting compiler and linker flags between our Python build and how we were building BSDDB (or rather how BSDDB builds out of the box on Windows).

My solution was to drop our reliance on the Berkeley_DB.sln/db_static.vcproj files completely, and mimic a bsddb44 vcproj in our own pcbuild.sln, which basically meant all the BSDDB source code got built in the exact same fashion as the rest of Python.  I also took this approach with sqlite3 and it's worked really well -- there have been no issues with either module since this change.

I've also got bsddb45.vcproj and bsddb46.vcproj projects floating around in one of my local branches somewhere.  These mimic the corresponding BSDDB projects, with the intent being that when it comes to release time for 2.6 and 3.0, we'd make a decision about which one to ship with, and then set the Python _bsddb module to use that.  I should probably pick that up again...

Hope this clarifies things...

Regards,

        Trent.

From martin at v.loewis.de  Tue Apr 22 21:58:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Apr 2008 21:58:08 +0200
Subject: [Python-Dev] 3k checkin mails to python-checkins
In-Reply-To: <fudo1l$snk$3@ger.gmane.org>
References: <fudo1l$snk$3@ger.gmane.org>
Message-ID: <480E4350.4060404@v.loewis.de>

> Since a few days, checkin notifications for the 3k branch seem to be sent
> to both the python-checkins and the python-3000-checkins lists. Was that a
> deliberate decision or has some bug crept into the SVN hook?

This should be fixed now. The new mailer.py had named some config
options differently from the old one.

Regards,
Martin

From mike.klaas at gmail.com  Tue Apr 22 22:46:26 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 22 Apr 2008 13:46:26 -0700
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480DBE86.2000507@egenix.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de> <480DBE86.2000507@egenix.com>
Message-ID: <1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>


On 22-Apr-08, at 3:31 AM, M.-A. Lemburg wrote:
>>>
>> I don't think that should be part of the standard library. People
>> will mistake what it tells them for certain.
>
> +1
>
> I also think that it's better to educate people to add (correct)
> encoding information to their text data, rather than give them a
> guess mechanism...

That is a fallacious alternative: the programmers that need encoding  
detection are not the same people who are omitting encoding information.

I only have a small opinion on whether charset detection should appear  
in the stdlib, but I am somewhat perplexed by the arguments in this  
thread.  I don't see how inclusion in the stdlib would make people  
more inclined to think that the algorithm is always correct.  In terms  
of the need of this functionality:

Martin wrote:
> Can you please explain why that is? Web programs should not normally
> have the need to detect the encoding; instead, it should be specified
> always - unless you are talking about browsers specifically, which
> need to support web pages that specify the encoding incorrectly.

Any program that needs to examine the contents of documents/feeds/ 
whatever on the web needs to deal with incorrectly-specified encodings  
(which, sadly, is rather common).  The set of programs of programs  
that need this functionality is probably the same set that needs  
BeautifulSoup--I think that set is larger than just browsers <grin>

-Mike

From mal at egenix.com  Tue Apr 22 22:54:35 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 22 Apr 2008 22:54:35 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <Pine.LNX.4.64.0804220636240.6873@apps3.cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de>	<480DBE86.2000507@egenix.com>	<Pine.LNX.4.64.0804220636240.6873@apps3.cs.toronto.edu>
Message-ID: <480E508B.2060006@egenix.com>

[CCing python-dev again]

On 2008-04-22 12:38, Greg Wilson wrote:
>>> I don't think that should be part of the standard library. People
>>> will mistake what it tells them for certain.
>>> [etc]
> 
> These are all good arguments, but the fact remains that we can't control 
> our inputs (e.g., we're archiving mail messages sent to lists managed by 
> DrProject), and some of those inputs *don't* tell us how they're encoded.
> Under those circumstances, what would you recommend?

I haven't done much research into this, but in general, I think it's
better to:

  * first try to look at other characteristics of a text
    message, e.g. language, origin, topic, etc.,

  * then narrow down the number of encodings which could apply,

  * rank them to try to avoid ambiguities and

  * then try to see what percentage of the text you can decode using
    each of the encodings in reverse ranking order (ie. more specialized
    encodings should be tested first, latin-1 last).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 22 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From asmodai at in-nomine.org  Tue Apr 22 23:12:18 2008
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Tue, 22 Apr 2008 23:12:18 +0200
Subject: [Python-Dev] Known doctest bug with unicode?
In-Reply-To: <aac2c7cb0804180905u3fc0216eie659aed72a8e0311@mail.gmail.com>
References: <20080418141216.GF64048@nexus.in-nomine.org>
	<361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>
	<20080418142705.GG64048@nexus.in-nomine.org>
	<aac2c7cb0804180905u3fc0216eie659aed72a8e0311@mail.gmail.com>
Message-ID: <20080422211218.GZ64048@nexus.in-nomine.org>

-On [20080418 18:05], Adam Olsen (rhamph at gmail.com) wrote:
>4. Make doctest smarter, so that it can grab the original module's encoding.
>5. Wait until 3.0, where this is hopefully fixed by making doctests
>use unicode by default?

Getting rid of the u in front of the strings as required made Python 3
indeed run the doctests as they should.

So there's a difference in behaviour between 2.x and 3.0 when it comes to
this part. I guess the better behaviour would be for doctest to honour the
encoding specified in the file/module? If other people agree I can see what
I can to make that work.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Confutatis maledictis, flammis acribus addictis...

From martin at v.loewis.de  Tue Apr 22 23:16:02 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 22 Apr 2008 23:16:02 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de>
	<480DBE86.2000507@egenix.com>
	<1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>
Message-ID: <480E5592.3050102@v.loewis.de>

>> Can you please explain why that is? Web programs should not normally
>> have the need to detect the encoding; instead, it should be specified
>> always - unless you are talking about browsers specifically, which
>> need to support web pages that specify the encoding incorrectly.
> 
> Any program that needs to examine the contents of
> documents/feeds/whatever on the web needs to deal with
> incorrectly-specified encodings

That's not true. Most programs that need to examine the contents of
a web page don't need to guess the encoding. In most such programs,
the encoding can be hard-coded if the declared encoding is not
correct. Most such programs *know* what page they are webscraping,
or else they couldn't extract the information out of it that they
want to get at.

As for feeds - can you give examples of incorrectly encoded one
(I don't ever use feeds, so I honestly don't know whether they
are typically encoded incorrectly. I've heard they are often XML,
in which case I strongly doubt they are incorrectly encoded)

As for "whatever" - can you give specific examples?

> (which, sadly, is rather common). The
> set of programs of programs that need this functionality is probably the
> same set that needs BeautifulSoup--I think that set is larger than just
> browsers <grin>

Again, can you give *specific* examples that are not web browsers?
Programs needing BeautifulSoup may still not need encoding guessing,
since they still might be able to hard-code the encoding of the web
page they want to process.

In any case, I'm very skeptical that a general "guess encoding"
module would do a meaningful thing when applied to incorrectly
encoded HTML pages.

Regards,
Martin

From martin at v.loewis.de  Tue Apr 22 23:32:55 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 22 Apr 2008 23:32:55 +0200
Subject: [Python-Dev] Known doctest bug with unicode?
In-Reply-To: <20080422211218.GZ64048@nexus.in-nomine.org>
References: <20080418141216.GF64048@nexus.in-nomine.org>	<361b27370804180717nce0cd15y2c131ac18478fb56@mail.gmail.com>	<20080418142705.GG64048@nexus.in-nomine.org>	<aac2c7cb0804180905u3fc0216eie659aed72a8e0311@mail.gmail.com>
	<20080422211218.GZ64048@nexus.in-nomine.org>
Message-ID: <480E5987.1030906@v.loewis.de>

> So there's a difference in behaviour between 2.x and 3.0 when it comes to
> this part. I guess the better behaviour would be for doctest to honour the
> encoding specified in the file/module? If other people agree I can see what
> I can to make that work.

I'm fairly skeptical that you can make that work, whether or not it's a
good idea.

Regards,
Martin


From mal at egenix.com  Tue Apr 22 23:34:20 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 22 Apr 2008 23:34:20 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <08Apr22.093328pdt."58696"@synergy1.parc.xerox.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de>
	<480DBE86.2000507@egenix.com>	<08Apr22.081453pdt."58696"@synergy1.parc.xerox.com>
	<08Apr22.093328pdt."58696"@synergy1.parc.xerox.com>
Message-ID: <480E59DC.3010007@egenix.com>

On 2008-04-22 18:33, Bill Janssen wrote:
> The 2002 paper "A language and character set determination method
> based on N-gram statistics" by Izumi Suzuki and Yoshiki Mikami and
> Ario Ohsato and Yoshihide Chubachi seems to me a pretty good way to go
> about this. 

Thanks for the reference.

Looks like the existing research on this just hasn't made it into the
mainstream yet.

Here's their current project: http://www.language-observatory.org/
Looks like they are focusing more on language detection.

Another interesting paper using n-grams:
"Language Identification in Web Pages" by Bruno Martins and M?rio J. Silva
http://xldb.fc.ul.pt/data/Publications_attach/ngram-article.pdf

And one using compression:
"Text Categorization Using Compression Models" by   	
Eibe Frank, Chang Chui, Ian H. Witten
http://portal.acm.org/citation.cfm?id=789742

> They're looking at "LSE"s, language-script-encoding
> triples; a "script" is a way of using a particular character set to
> write in a particular language.
> 
> Their system has these requirements:
> 
> R1. the response must be either "correct answer" or "unable to detect"
>     where "unable to detect" includes "other than registered" [the
>     registered set of LSEs];
> 
> R2. Applicable to multi-LSE texts;
> 
> R3. never accept a wrong answer, even when the program does not have
>     enough data on an LSE; and
> 
> R4. applicable to any LSE text.
> 
> So, no wrong answers.
> 
> The biggest disadvantage would seem to be that the registration data
> for a particular LSE is kind of bulky; on the order of 10,000
> shift-codons, each of three bytes, about 30K uncompressed.
> 
> http://portal.acm.org/ft_gateway.cfm?id=772759&type=pdf

For a server based application that doesn't sound too large.

Unless you're using a very broad scope, I don't think that
you'd need more than a few hundred LSEs for a typical
application - nothing you'd want to put in the Python stdlib,
though.

> Bill
> 
>>> IMHO, more research has to be done into this area before a
>>> "standard" module can be added to the Python's stdlib... and
>>> who knows, perhaps we're lucky and by the time everyone is
>>> using UTF-8 anyway :-)
>> I walked over to our computational linguistics group and asked.  This
>> is often combined with language guessing (which uses a similar
>> approach, but using characters instead of bytes), and apparently can
>> usually be done with high confidence.  Of course, they're usually
>> looking at clean texts, not random "stuff".  I'll see if I can get
>> some references and report back -- most of the research on this was
>> done in the 90's.
>>
>> Bill

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 22 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From alessandro.guido at gmail.com  Tue Apr  8 15:15:24 2008
From: alessandro.guido at gmail.com (Alessandro Guido)
Date: Tue, 8 Apr 2008 15:15:24 +0200
Subject: [Python-Dev] py3k: print function treats sep=None and end=None in
	an unintuitive way
Message-ID: <200804081515.24887.alessandro.guido@gmail.com>

Can anybody please point me why print('a', 'b', sep=None, end=None) should
produce "a b\n" instead of "ab"?
I've read http://docs.python.org/dev/3.0/library/functions.html#print, pep-3105 and some
ml threads but did not find a good reason justifying such a strange behaviour.

Thanks.

	-Alessandro Guido

From coughlan at ski.org  Tue Apr  8 19:13:06 2008
From: coughlan at ski.org (James Coughlan)
Date: Tue, 08 Apr 2008 10:13:06 -0700
Subject: [Python-Dev]  New project : Spyke python-to-C compiler
Message-ID: <47FBA7A2.8000701@ski.org>

Hello,

Your project sounds very interesting! I would be happy to send you some 
test code -- perhaps you could give me some more details about what 
you're looking for. (E.g. *small,* self-contained python/numpy functions.)

Also, are you aware of Shed Skin? "Shed Skin is an experimental 
Python-to-C++ compiler. It can convert pure, but implicitly statically 
typed Python programs into optimized C++ code."
http://shed-skin.blogspot.com/

Best,

James


-- 
-------------------------------------------------------
James Coughlan, Ph.D., Scientist                     

The Smith-Kettlewell Eye Research Institute

Email: coughlan at ski.org
URL: http://www.ski.org/Rehab/Coughlan_lab/
Phone: 415-345-2146 
Fax: 415-345-8455
-------------------------------------------------------


From python at venix.com  Tue Apr  8 20:27:39 2008
From: python at venix.com (Lloyd Kvam)
Date: Tue, 08 Apr 2008 14:27:39 -0400
Subject: [Python-Dev] [Distutils] how to easily consume just the
	parts	of eggs that are	 good for you
In-Reply-To: <73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
Message-ID: <1207679260.3474.74.camel@localhost.localdomain>

On Tue, 2008-04-08 at 10:01 -0700, zooko wrote:
> They both agreed that it made perfect sense.  I told one of them  
> about the alternate proposal to define a new database file to
> contain  
> a list of installed packages, and he sighed and rolled his eyes and  
> said "So they are planning to reinvent apt!".

When I wear my sysadmin hat, eggs become a nuisance.  They are not
listed in the system packages; if zipped they won't work when the apache
user tries to import them; easy_install can produce unexpected upgrades.
The system package manager (apt or yum) is much preferred.

As a developer, eggs are great.  If a python module is not already
available from my system packagers, easy_install will find it, get it,
and install it.  I waste almost no time with system administration
issues while developing.

Fortunately, distutils includes tools like bdist_rpm so that python
modules can be packaged for easy processing by the system package
manager.  So once I need to switch back to a sysadmin role, I can use
the system tools to install and track packages.

-- 
Lloyd Kvam
Venix Corp
DLSLUG/GNHLUG library
http://www.librarything.com/catalog/dlslug
http://www.librarything.com/profile/dlslug
http://www.librarything.com/rsshtml/recent/dlslug


From gael.varoquaux at normalesup.org  Wed Apr  9 09:54:16 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 9 Apr 2008 09:54:16 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that	are good for you
In-Reply-To: <87bq4jolks.fsf@benfinney.id.au>
References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com>
	<47EB07A6.6040800@plope.com>
	<73387C02-3D5C-4D9E-A886-82D486AC0C76@zooko.com>
	<87bq4jolks.fsf@benfinney.id.au>
Message-ID: <20080409075416.GA1761@phare.normalesup.org>

On Wed, Apr 09, 2008 at 11:37:07AM +1000, Ben Finney wrote:
> zooko <zooko at zooko.com> writes:

> > I am skeptical that prorgammers are going to be willing to use a new
> > database format. They already have a database -- their filesystem --
> > and they already have the tools to control it -- mv, rm, and
> > PYTHONPATH. Many of them already hate the existence the
> > "easy_instlal.pth" database file, and I don't see why a new database
> > file would be any different.

> Moreover, many of us already have a database of *all* packages on the
> system, not just Python-language ones: the package database of our
> operating system. Adding another, parallel, database which needs
> separate maintenance, and only applies to Python packages, is not a
> step forward in such a situation.

90 % (at least) of the world does not have such database. I, and probably
you, have such a very nice database. I works well, and we can choose to
forget the problems our users are facing. It does not solve them though.

In addition, packaging is system-specific. I recently had to learn some
Debian packaging, because I wanted my Ubuntu and Debian users to be able
to use my projects seamlessly. What about RPMs for RHEL, Fedora,
Mandriva? ... and coronary packages? and MSIs? ... When do I find time to
do development if I have to learn all this packaging.

It would be fantastic to have an abstraction on all these packaging
systems, including, as you point out, their database. I do agree that
reusing the system packaging's database is great, and would be the best
option for system-wide install. However one of the very neat features of
setuptools and eggs is that you don't need administrator access to
install the packages, and that is great in a shared environment, like a
computation cluster. The system's database is thus unfortunately not a
complete solution to the problem.

My 2 cents,

Ga?l

From gael.varoquaux at normalesup.org  Wed Apr  9 10:00:16 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 9 Apr 2008 10:00:16 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that	are good for you
In-Reply-To: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
Message-ID: <20080409080016.GB1761@phare.normalesup.org>

On Wed, Apr 09, 2008 at 12:41:32AM -0400, Phillip J. Eby wrote:
> >The way to achieve a database for Python would be to provide tools for
> >conversion of eggs to rpms and debs,

> Such tools already exist, although the conversion takes place from 
> source distributions rather than egg distributions.

What is the status of the deb backend? The only one I know is unofficial
maintained by Andrew Straw, but my information my be lagging behind.

By the way, if these tools work well, they are priceless!

Cheers,

Ga?l

From sklein at cpcug.org  Wed Apr  9 17:52:53 2008
From: sklein at cpcug.org (Stanley A. Klein)
Date: Wed, 9 Apr 2008 11:52:53 -0400 (EDT)
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
Message-ID: <33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>


On Wed, April 9, 2008 12:41 am, Phillip J. Eby wrote:
> At 10:49 PM 4/8/2008 -0400, Stanley A. Klein wrote:
>>On Tue, April 8, 2008 9:37 pm, Ben Finney
>><bignose+hates-spam at benfinney.id.au> wrote:
>> > Date: Wed, 09 Apr 2008 11:37:07 +1000
>> > From: Ben Finney <bignose+hates-spam at benfinney.id.au>
>> > Subject: Re: [Distutils] how to easily consume just the parts of eggs
>> >       that    are     good for you
>> > To: Distutils-Sig at Python.Org
>> >
>> >
>> > zooko <zooko at zooko.com> writes:
>> >> eyes and said "So they are planning to reinvent apt!".
>> >
>> > That's pretty much my reaction, too.
>>
>>I have the same reaction.
>
> I'm curious.  Have any of you actually read PEP 262 in any detail?  I
> have seen precious little discussion so far that doesn't appear to be
> based on significant misunderstandings of either the purpose of
> reviving the PEP, or the mechanics of its proposed implementation.

I haven't read the PEP at all.  I generally don't read PEP's.

>
>
>>I have tried in the past to use easy_install, but have run into problems
>>because there is no communication between easy_install and the rpm
>>database, resulting in failure of easy_install to recognize that
>>dependencies have already been installed using rpms.
>
> This problem doesn't exist with Python 2.5, unless you're using a
> platform that willfully strips out the installation information that
> Python 2.5 provides for these packages.
>

IIRC, I have had the problem with Python 2.5 on Fedora 7.  Until recently,
Fedora packagers did strip out the egg information included with Python
packages they packaged.  I left those files in when packaging myself using
bdist_rpm.

However, are you implying that the installation information for Python egg
packages accesses and coordinates with the rpm database?  I found myself
having to go into the setup.py for the relevant package(s) and delete any
statements regarding dependencies.  Otherwise, IIRC, the packaging
couldn't proceed because the Python packaging tool couldn't find the
dependencies that had already been installed as rpms.  After installation,
Python managed to find the relevant files, but the packaging tool
couldn't.

>
>>A database focused only on Python packages is highly inappropriate for
>>Linux systems, violates the Linux standards, and creates problems because
>>eggs are not coordinated with the operating system package manager.
>
> The revamp of PEP 262 is aimed at removing .egg files and directories
> from the process, by allowing system packagers to tell Python what
> files belong to them and should not be messed with.  And conversely,
> allowing systems and installation targets *without* package managers
> to safely manage their Python installations.

IMHO, the main system without a package manager is Windows.  A reasonable
way to deal with Windows would be to create a package manager for it that
could be used by Python and anyone else who wanted to use it.  The package
manager could establish a file hierarchy similar to the Unix FHS and
install files appropriately, except for what is needed to satisfy the
Windows OS.  That would probably go a long way to addressing the issues
being discussed here.  This is primarily a Windows problem, not a Python
problem.


>
>
>>   The
>>way to achieve a database for Python would be to provide tools for
>>conversion of eggs to rpms and debs,
>
> Such tools already exist, although the conversion takes place from
> source distributions rather than egg distributions.
>

You are talking here about bdist_rpm and not about a tool that would take
a Python package distributed as an egg file and convert the egg to an rpm
or a deb.  Unfortunately, some Python packagers are beginning to limit
their focus only to egg distribution.  That creates a problem for users
who have native operating system package management.

>
>>to have eggs support conformance to
>>the LSB and FHS,
>
> Applying LSB and FHS to the innards of Python packages makes as much
> sense as applying them to the contents of Java .jar files -- i.e.,
> none.  If it's unchanging data that's part of a program or library,
> then it's a program or library, just like static data declared in a C
> program or library.  Whether the file extension is .py, .so, or even
> .png is irrelevant.

The FHS defines places to put specific kinds of files, such as command
scripts (/bin, /usr/bin, /sbin, or /usr/sbin), documentation
(/usr/share/doc/package-name), and configuration files (/etc).  There are
several kinds of files identified and places defined to put them. 
Distribution by eggs has a tendency to scoop up all of those files and put
them in /usr/lib/python/site-packages, regardless of where they belong. 
Having eggs support conformance to FHS would mean recognizing and tagging
the relevant files.  A tool for converting eggs to rpms or debs would
essentially reformat the egg to rpm or deb and put files where they
belong.


Stan Klein




From brianc at temple.edu  Wed Apr  9 19:36:38 2008
From: brianc at temple.edu (Brian Cole)
Date: Wed, 9 Apr 2008 11:36:38 -0600
Subject: [Python-Dev] Python Leopard DLL Hell
In-Reply-To: <47FC2161.5020905@gmail.com>
References: <54b165660804081819g4787f4d9g821c7269af7b4c66@mail.gmail.com>
	<47FC2161.5020905@gmail.com>
Message-ID: <54b165660804091036n40aef4c9r98aabe2a059fc19e@mail.gmail.com>

I have learned that this is a specific behavior of OS X. I have
submitted a formal bug report to Apple about the problem. It appears
that this is documented by Apple as acceptable:
http://developer.apple.com/documentation/DeveloperTools/Reference/MachOReference/Reference/reference.html#//apple_ref/c/func/dlopen

Whereas, linux will respect the fact you gave it a specific shared library:
http://linux.die.net/man/3/dlopen

If I am provided a workaround by apple I will post a python patch. A
little scary that someone can circumvent my application by just
setting an environment variable.

-Brian Cole

On Tue, Apr 8, 2008 at 7:52 PM, Michael Torrie <torriem at gmail.com> wrote:
> Brian Cole wrote:
>  > That appears to be working correctly at first glance. The argument to
>  > dlopen is the correct shared library. Unfortunately, either python or
>  > OS X is lying to me here. If I inspect the python process with OS X's
>  > Activity Monitor and look at the "Open Files and Ports" tab, it shows
>  > that the _foo.so shared library is actually the one located inside
>  > $DYLD_LIBRARY_PATH.
>  >
>  > So this problem may not be python's, but I place it here as a first
>  > shot (maybe I'm using the imp module incorrectly).
>
>  Sounds like you're going to need to learn how to use dtrace.  Then you
>  can more closely monitor exactly what python and the loader are doing.
>  dtrace is very complicated (borrowed from Solaris) but extremely
>  powerful.  Worth learning anyway, but sounds like it's probably the
>  debugging tool you need.
>
>  Another thing you can do is check through the python source code and see
>  how the os x-specific code is handling this type of situation.
>
>  --
>  http://mail.python.org/mailman/listinfo/python-list
>

From sklein at cpcug.org  Wed Apr  9 20:26:31 2008
From: sklein at cpcug.org (Stanley A. Klein)
Date: Wed, 9 Apr 2008 14:26:31 -0400 (EDT)
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
Message-ID: <36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>

All my development is done on Linux.  I use Windows very minimally (such
as for tax preparation) and unless forced to do so for specific
circumstances (such as submittal to grants.gov) do not expose Windows to
the Internet.

In the future there may possibly arise a need for us to port some
Linux-developed Python code to Windows, but we will have to cross that
bridge when we get there.

I think you raise an interesting issue:  What is a package manager?  I
have minimal experience installing packages on Windows over the last 5-10
years, but in my experience a Windows package comes as an executable that,
when run, installs itself.  Unless a third-party program monitors the
installation, uninstalling is a nasty chore, as is finding out what files
were installed or where they went.

The rpm and deb package managers (and their yum and other higher level
dependency managers) do a lot of things:

1.  They install packages and maintain databases of what packages were
installed
2.  They manage dependencies
3.  They support clean uninstalling of packages
4.  They can query packages, both installed (via their databases) and not
yet installed (e.g., as rpm or deb files), to determine attributes, such
as files they install, dependencies, and other information defined at
packaging time.
5.  They build packages and (in some cases) can rebuild packages.
6.  They can verify packages for integrity and security purposes.
7.  They can download package files and maintain archives of installed
package files for use as local repositories.

There may be other functions, but the above is a top-of-the-head list.

I can say that I'm not terribly happy with Python packaging that is only
minimally compatible with rpm.  I haven't used Ubuntu all that much. I do
like Ubuntu's packaging and package management, and I do know that there
are  programs, such as alias, that can translate from rpm to deb formats.

I don't think I ever said that Windows is broken in the area of package
management.  My own experience is that the files of Windows programs tend
to be put in a directory devoted to the program, rather than put in
directories with other files having similar purposes.  At one time, the
default location in Windows for word processing files was even in a
sub-directory of the word processing program.  That changed to having a
form of user home directory, but it didn't change much for the program
files themselves.  Unix/Linux puts the files in specific areas of the file
system having functional commonality.  One could almost say that the
Windows default approach to structuring its filesystem avoids or minimizes
the need for package management.

I repeat that this issue mainly arises because Windows doesn't have the
same kind of filesystem structure (and therefore the need for package
management) that other systems have.  I don't know what Windows add/remove
programs function does, but all it might do is to run the executable to
install packages and record the installation (as was previously done by
third party programs) to facilitate clean removal.  Unless you can perform
more of the other functions I listed above, I doubt I would call
add/remove a package manager.


Stan Klein



On Wed, April 9, 2008 1:23 pm, Paul Moore wrote:
> On 09/04/2008, Stanley A. Klein <sklein at cpcug.org> wrote:
>> IMHO, the main system without a package manager is Windows.  A
>> reasonable
>>  way to deal with Windows would be to create a package manager for it
>> that
>>  could be used by Python and anyone else who wanted to use it.  The
>> package
>>  manager could establish a file hierarchy similar to the Unix FHS and
>>  install files appropriately, except for what is needed to satisfy the
>>  Windows OS.  That would probably go a long way to addressing the issues
>>  being discussed here.  This is primarily a Windows problem, not a
>> Python
>>  problem.
>
> Windows does have a package manager - the add/remove programs
> application. It's extremely limited, and doesn't make any attempt at
> doing dependency resolution, certainly - but that's a separate issue.
>
> I don't know if you use Windows (as in, develop programs using Python
> on Windows). If you do, then I'd be interested in your views on
> bdist_wininst and bdist_msi installers, and how they fit into the
> setuptools/egg environment, particularly with regard to the package
> manager you are proposing. If you don't use Windows, then I don't see
> how you can usefully comment.
>
> Personally, as I've said before, I don't have a problem with a
> Python-only package manager, as long as it replaces or integrates
> bdist_wininst and bdist_msi. Having two package managers is far worse
> than having none - and claiming that add/remove programs "isn't a
> package manager" is just ignoring reality (if it isn't, then why do
> bdist_wininst and bdist_msi exist?).
>
> Are the Linux users happy with having a Python package manager that
> ignores RPM/apt? Why should Windows users be any happier?
>
> Sorry - I'm feeling a little grumpy. I've read one too many "Windows
> is so broken that people who use it obviously don't care about doing
> things right" postings this week :-(
>
> Paul.
>


-- 



From gael.varoquaux at normalesup.org  Wed Apr  9 21:19:48 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 9 Apr 2008 21:19:48 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that	are good for you
In-Reply-To: <36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
	<36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
Message-ID: <20080409191948.GA11856@phare.normalesup.org>

On Wed, Apr 09, 2008 at 02:26:31PM -0400, Stanley A. Klein wrote:
> The rpm and deb package managers (and their yum and other higher level
> dependency managers) do a lot of things:

> 1.  They install packages and maintain databases of what packages were
> installed
> 2.  They manage dependencies
> 3.  They support clean uninstalling of packages
> 4.  They can query packages, both installed (via their databases) and not
> yet installed (e.g., as rpm or deb files), to determine attributes, such
> as files they install, dependencies, and other information defined at
> packaging time.
> 5.  They build packages and (in some cases) can rebuild packages.
> 6.  They can verify packages for integrity and security purposes.
> 7.  They can download package files and maintain archives of installed
> package files for use as local repositories.

You are collapsing three different functionalities in one:

  * Dealing with repositories and downloading: yum/apt

 * Installing + uninstalling packages, and dealing with system
   consistency (thus checking the dependencies are available): rpm/dpkg

  * Building

For me it is important that the 3 are separated:

  * I may want to download the dependencies of a package to burn to a CD
    for a computer that does not have internet access.

  * I may want to send a tarball to a build server that does the building,
    but no install (so as not to corrupt my working system).

Cheers,

Ga?l

From sklein at cpcug.org  Wed Apr  9 22:00:18 2008
From: sklein at cpcug.org (Stanley A. Klein)
Date: Wed, 9 Apr 2008 16:00:18 -0400 (EDT)
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409191948.GA11856@phare.normalesup.org>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
	<36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
	<20080409191948.GA11856@phare.normalesup.org>
Message-ID: <12119.207.188.248.157.1207771218.squirrel@www.cpcug.org>


On Wed, April 9, 2008 3:19 pm, Gael Varoquaux wrote:
> On Wed, Apr 09, 2008 at 02:26:31PM -0400, Stanley A. Klein wrote:
>> The rpm and deb package managers (and their yum and other higher level
>> dependency managers) do a lot of things:
>
>> 1.  They install packages and maintain databases of what packages were
>> installed
>> 2.  They manage dependencies
>> 3.  They support clean uninstalling of packages
>> 4.  They can query packages, both installed (via their databases) and
>> not
>> yet installed (e.g., as rpm or deb files), to determine attributes, such
>> as files they install, dependencies, and other information defined at
>> packaging time.
>> 5.  They build packages and (in some cases) can rebuild packages.
>> 6.  They can verify packages for integrity and security purposes.
>> 7.  They can download package files and maintain archives of installed
>> package files for use as local repositories.
>
> You are collapsing three different functionalities in one:
>
>   * Dealing with repositories and downloading: yum/apt
>
>  * Installing + uninstalling packages, and dealing with system
>    consistency (thus checking the dependencies are available): rpm/dpkg
>
>   * Building
>
> For me it is important that the 3 are separated:
>
>   * I may want to download the dependencies of a package to burn to a CD
>     for a computer that does not have internet access.
>
>   * I may want to send a tarball to a build server that does the building,
>     but no install (so as not to corrupt my working system).
>
> Cheers,
>
> Ga?l
>

Gael -

The functionalities are combined in programs but are not necessarily
required to be used all at the same time.

I'm not that familiar with apt, but yum also installs, including
downloading both a package and its dependencies.  Yum also has a query
capability (yum list, yum info).  I think synaptic does the same thing yum
does, and adds a GUI and search capabilities similar to yum info as well.

The build capabilities of rpm were moved to rpmbuild, but the building
remains part of the rpm system.  IIRC, bdist_rpm actually calls rpmbuild
as part of its processing.

Also, IIRC, rpmbuild can build from a tarball if it contains an rpm spec. 
It does not install in the same process.  That is a separate step.  You
would not corrupt your working system by building an rpm from a tarball on
it.

BTW, I would not want to do dependencies with rpm if yum is available. 
Doing dependencies with rpm is very difficult and it is easy to wind up in
"dependency hell".  Yum will find the dependencies and install them as
long as they are in repositories that are registered in the yum
configuration.

I looked at "man yum" and couldn't find an option to download dependencies
to the local repository without installing.  However, if you did install a
package and its dependencies, and if you have selected the option of
retaining the cache and not cleaning it after installation, the rpms
(e.g., for updates) are in /var/cache/yum/updates/packages/.  They can be
copied from there to a CD for a system without internet connectivity. 
Also, both Fedora and Ubuntu have software for building installable live
CD's, although I don't know how they get their package files.


Stan Klein



From sklein at cpcug.org  Wed Apr  9 22:43:32 2008
From: sklein at cpcug.org (Stanley A. Klein)
Date: Wed, 9 Apr 2008 16:43:32 -0400 (EDT)
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <20080409194044.1A4E03A406A@sparrow.telecommunity.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com>
Message-ID: <12121.207.188.248.157.1207773812.squirrel@www.cpcug.org>

On Wed, April 9, 2008 3:40 pm, Phillip J. Eby wrote:
> At 11:52 AM 4/9/2008 -0400, Stanley A. Klein wrote:
>>However, are you implying that the installation information for Python
>> egg
>>packages accesses and coordinates with the rpm database?
>
> Yes, when the information isn't stripped out.  Try a more recent Fedora.
>
>
>>IMHO, the main system without a package manager is Windows.
>
> You're ignoring shared environments and development
> environments.  (Not to mention Mac OS.)
>

I don't understand what you mean by "shared environments and development
 environments".  I also don't know much about Mac OS, except that its
underlying Darwin system is a version of BSD (that I assume would follow
the Unix FHS).


>
>>   A reasonable
>>way to deal with Windows would be to create a package manager for it that
>>could be used by Python and anyone else who wanted to use it.
>
> Let us know when you've finished it, along with the one for Mac OS.  :)

I have enough trouble with what I'm already doing.  :-)

>
> Of course this still won't do anything for shared environments and
> development environments.
>
>
>>You are talking here about bdist_rpm and not about a tool that would take
>>a Python package distributed as an egg file and convert the egg to an rpm
>>or a deb.  Unfortunately, some Python packagers are beginning to limit
>>their focus only to egg distribution.  That creates a problem for users
>>who have native operating system package management.
>
> That is indeed a problem -- but it's a social one, not a technical
> one.  It's trivial for the publisher of an egg to change their
> command line from "setup.py bdist_egg upload" to "setup.py sdist
> bdist_egg upload", as soon as their users (politely) request that they do
> so.
>

I agree that we are dealing with a combination of technical and social
issues here.  However, I think it takes a lot more understanding for a
publisher to get everything straight.

>
>> > Applying LSB and FHS to the innards of Python packages makes as much
>> > sense as applying them to the contents of Java .jar files -- i.e.,
>> > none.  If it's unchanging data that's part of a program or library,
>> > then it's a program or library, just like static data declared in a C
>> > program or library.  Whether the file extension is .py, .so, or even
>> > .png is irrelevant.
>>
>>The FHS defines places to put specific kinds of files, such as command
>>scripts (/bin, /usr/bin, /sbin, or /usr/sbin), documentation
>>(/usr/share/doc/package-name), and configuration files (/etc).  There are
>>several kinds of files identified and places defined to put them.
>>Distribution by eggs has a tendency to scoop up all of those files and
>> put
>>them in /usr/lib/python/site-packages, regardless of where they belong.
>
> Eggs don't include documentation or configuration files, and they
> install scripts in script directories, so I don't get what you're
> talking about here.  For any other data that a package accesses at
> runtime, my earlier comments apply.
>

But rpms and debs do include these files, plus manual pages, localization
files and a lot of other ancillary stuff.

IIRC, you once mentioned that you have a CENTOS system.  Do an "rpm -qa
|sort|less" to get an alphabetized list of your installed packages, and
then an "rpm -qil" on some of the packages, and you will see the range of
different kinds of files in there.

>
>>Having eggs support conformance to FHS would mean recognizing and tagging
>>the relevant files.  A tool for converting eggs to rpms or debs would
>>essentially reformat the egg to rpm or deb and put files where they
>>belong.
>
> No, because such files as you describe don't exist.  If you think
> they do, then either you have misunderstood the nature of the files
> in question, or the developer has incorrectly placed non-runtime
> files in their installation tree.
>

Most of the Python tarballs I have downloaded have all kinds of files in
their installation trees.  This is a major pain in the you-know-what for
someone trying to use bdist_rpm and get proper, FHS-compliant rpms.  If
eggs are supposed to be strictly runtime files, I think very few
developers actually understand that.  Better yet, how do you define what
should be included in an installation?  It sounds like the egg concept
doesn't include several kinds of files that rpm and deb would include in
an installation.  I think that may be an important issue here.


Stan Klein



From gael.varoquaux at normalesup.org  Thu Apr 10 00:51:38 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 10 Apr 2008 00:51:38 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that	are good for you
In-Reply-To: <79990c6b0804091546p5cf67f2ta800c675e7baa256@mail.gmail.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<79990c6b0804091023p51a0e343l73ac695944a4c9be@mail.gmail.com>
	<36773.207.188.248.157.1207765591.squirrel@www.cpcug.org>
	<79990c6b0804091546p5cf67f2ta800c675e7baa256@mail.gmail.com>
Message-ID: <20080409225138.GH28763@phare.normalesup.org>

On Wed, Apr 09, 2008 at 11:46:19PM +0100, Paul Moore wrote:
> I find this whole discussion hugely confusing, because a lot of people
> are stating opinions about environments which it seems they don't use,
> or know much about. I don't know how to avoid this, but it does make
> it highly unlikely that any practical progress will get made.

I find that something that doesn't help at all the discussion move
forward is that everybody has different usecases in mind, on different
platforms, and is not interested in other people's usecases.

Hopefuly I am wrong,

Cheers,

Ga?l

From gael.varoquaux at normalesup.org  Thu Apr 10 00:59:24 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 10 Apr 2008 00:59:24 +0200
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
	of eggs that	are good for you
In-Reply-To: <79990c6b0804091552h4ee7f9f6t5512ed5f5aa70d2a@mail.gmail.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com>
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org>
	<87iqyqir2t.fsf@benfinney.id.au>
	<79990c6b0804091552h4ee7f9f6t5512ed5f5aa70d2a@mail.gmail.com>
Message-ID: <20080409225924.GI28763@phare.normalesup.org>

On Wed, Apr 09, 2008 at 11:52:08PM +0100, Paul Moore wrote:
> And I would say that Windows doesn't have a problem. Are any Windows
> users proposing building a package management system for Windows
> (Python-specific or otherwise)? It's a genuine question - is this
> something that Windows users are after, or is it just Linux users
> trying to show Windows users what they are missing?

Well, users don't phrase this that way, because they don't know what
package management (or rather automatic dependency tracking) is, but yes,
they are some usecases. It is nowadays really tedious to deploy Python
applications making uses of many packages on Python.

The scientific community is a domain in which this problem is crucial, as
we are trying to ship desktop applications to non-computer-savy people,
with many dependencies outside the standard library.

Enthought is working on shipping a Python distribution with some sort of
package management for this purpose ( see
http://code.enthought.com/enstaller/ ), and finding it is not an easy
problem.

Cheers,

Gael

From adelagon at gmail.com  Thu Apr 10 03:23:39 2008
From: adelagon at gmail.com (Alvin Delagon)
Date: Thu, 10 Apr 2008 09:23:39 +0800
Subject: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
Message-ID: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>

Hello fellow pythonistas,

I'm currently writing a simple python SCTP module in C. So far it works
sending and receiving strings from it. The C sctp function sctp_sendmsg()
has been wrapped and my function looks like this:

sendMessage(PyObject *self, PyObject *args)
{
  const char *msg = "";
  if (!PyArg_ParseTuple(args, "s", &msg))
    return NULL;
  snprintf(buffer, 1025, msg);
  ret = sctp_sendmsg(connSock, (void *)buffer, (size_t)strlen(buffer), 0, 0,
0x03000000, 0, 0, 0, 0);
  return Py_BuildValue("b", "");
}

I'm going to construct an SS7 packet in python using struct.pack(). Here's
the question, how am I going to pass the packet I wrote in python to my
module and back? I already asked this question in comp.lang.python but so
far no responses yet. I hope anyone can point me to the right direction.
Thanks in advance.

---
Alvin Delagon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080410/db12075f/attachment-0001.htm>

From sklein at cpcug.org  Thu Apr 10 03:58:57 2008
From: sklein at cpcug.org (Stanley A. Klein)
Date: Wed, 9 Apr 2008 21:58:57 -0400 (EDT)
Subject: [Python-Dev] [Distutils] how to easily consume just the parts
 of eggs that are good for you
In-Reply-To: <47FD4E9D.8060600@enthought.com>
References: <20080409044125.AB5153A40D7@sparrow.telecommunity.com> 
	<33116.207.188.248.157.1207756373.squirrel@www.cpcug.org> 
	<20080409194044.1A4E03A406A@sparrow.telecommunity.com> 
	<47FD4E9D.8060600@enthought.com>
Message-ID: <43869.71.178.83.244.1207792737.squirrel@www.cpcug.org>

On Wed, 2008-04-09 at 18:17 -0500, Dave Peterson wrote:
>   I think I can sum up any further points by simply asking: "Should it
> be safe to assume I can distribute my application via eggs /
> easy_install just because it is written in Python?"


I think that based on this discussion the bottom line answer to this
question is "No".


Stan Klein



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080409/ac2a1381/attachment-0001.htm>

From lclark at 2bdx.net  Tue Apr 15 22:32:12 2008
From: lclark at 2bdx.net (Leonard Clark)
Date: Tue, 15 Apr 2008 13:32:12 -0700
Subject: [Python-Dev] Patch submitted for xmlrpclib
Message-ID: <AADFF021-9B21-4E32-B581-260150371267@2bdx.net>

I submitted a patch a few days ago (http://bugs.python.org/issue2623)  
to fix a datetime parameter formatting issue (See issue)  I was  
wondering if this was adequate and whether it could be included in  
future releases.

Thank you.

--
Leonard Clark




From ikshefem at gmail.com  Wed Apr 16 15:08:38 2008
From: ikshefem at gmail.com (iks hefem)
Date: Wed, 16 Apr 2008 15:08:38 +0200
Subject: [Python-Dev] SetType=set in types module ?
Message-ID: <78e18a710804160608j68a07056y63fcd3a0ebab08ca@mail.gmail.com>

Hi,

the SetType is not available in the "types" module, so wouldn't it be
needed here ? (in 2.6 by example)

I guess the change is really simple and would be backward compatible :
adding SetType = set

From tom at vector-seven.com  Thu Apr 17 01:23:53 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Thu, 17 Apr 2008 09:23:53 +1000
Subject: [Python-Dev] Global Python Sprint Weekends: May 10th-11th and
 June	21st-22nd.
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E22F389FE@EXMBX04.exchhosting.com>
Message-ID: <48068A89.9000306@vector-seven.com>

Anyone in Melbourne, Australia keen for the first sprint? I'm not sure 
if I'll be available, but if I can it'd be great to work with some 
others. Failing that, it's red bull and pizza in my lounge room :)

I've been working on some neat code for an AST optimizer. If I'm free 
that weekend, I'll probably continue my work on that.

Cheers,
T

Trent Nelson wrote:
>     Following on from the success of previous sprint/bugfix weekends and
>     sprinting efforts at PyCon 2008, I'd like to propose the next two
>     Global Python Sprint Weekends take place on the following dates:
>
>         * May 10th-11th (four days after 2.6a3 and 3.0a5 are released)
>         * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released)
>
>     It seems there are a few of the Python User Groups keen on meeting
>     up in person and sprinting collaboratively, akin to PyCon, which I
>     highly recommend.  I'd like to nominate Saturday across the board
>     as the day for PUGs to meet up in person, with Sunday geared more
>     towards an online collaboration day via IRC, where we can take care
>     of all the little things that got in our way of coding on Saturday
>     (like finalising/preparing/reviewing patches, updating tracker and
>     documentation, writing tests ;-).
>
>     For User Groups that are planning on meeting up to collaborate,
>     please reply to this thread on python-dev at python.org and let every-
>     one know your intentions!
>
>     As is commonly the case, #python-dev on irc.freenode.net will be
>     the place to be over the course of each sprint weekend; a large
>     proportion of Python developers with commit access will be present,
>     increasing the amount of eyes available to review and apply patches.
>
>     For those that have an idea on areas they'd like to sprint on and
>     want to look for other developers to rope in (or just to communicate
>     plans in advance), please also feel free to jump on this thread via
>     python-dev@ and indicate your intentions.
>
>     For those that haven't the foggiest on what to work on, but would
>     like to contribute, the bugs tracker at http://bugs.python.org is
>     the best place to start.  Register an account and start searching
>     for issues that you'd be able to lend a hand with.
>
>     All contributors that submit code patches or documentation updates
>     will typically get listed in Misc/ACKS.txt; come September when the
>     final release of 2.6 and 3.0 come about, you'll be able to point at
>     the tarball or .msi and exclaim loudly ``I helped build that!'',
>     and actually back it up with hard evidence ;-)
>
>     Bring on the pizza and Red Bull!
>
>         Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/krumms%40gmail.com
>   


From robert.hoelzl at baltech.de  Thu Apr 17 12:08:41 2008
From: robert.hoelzl at baltech.de (=?ISO-8859-15?Q?Robert_H=F6lzl?=)
Date: Thu, 17 Apr 2008 12:08:41 +0200
Subject: [Python-Dev] socket recv on win32 can be extremly delayed (python
	bug?)
Message-ID: <480721A9.1070603@baltech.de>

hello,

I tried to implement a simple python XMLRPC service on a win32 
environment (client/server code inserted below).
The profiler of the client told me, that a simple function call needs 
about 200ms (even if I run it in a loop, the time needed per call stays 
the same).

After analysing the problem with etherreal I found out, that the XMLRPC 
request is transmitted via two TCP packets. One containing the HTTP 
header and one containting the data. But the acknowledge to the first 
TCP packet is delayed by 200ms.

I tried around on the server side and found out that if the server reads 
exactly all bytes transfered in the first TCP frame (via socket.recv()), 
the next socket.recv(), even if reading only one byte, needs about 200 
ms. But if I read one byte less than transfered in the first TCP frame 
and then reading 2 bytes (socket.recv(2)) there is no delay, although 
the same total amount of data was read.

After some googling I found the website 
http://support.microsoft.com/?scid=kb%3Ben-us%3B823764&x=12&y=15, which 
proposed a workaround (modifing the registryentry for the tcp/ip driver) 
that did work. But modifing the clients registry settings is no option 
for us.

Is there anybody who nows how to solve the problem? Or is it even a 
problem if the python socket implementation?

By the way: I testet Win2000 SP4 and WinXP SP2 with Python 2.3.3 and 
Python 2.5.1 each.

CLIENT:
----------
import xmlrpclib
import profile
server = xmlrpclib.ServerProxy("http://server:80")
profile.run('server.test(1,2)')

SERVER:
----------
import SimpleXMLRPCServer
def test(a,b): return a+b
server = SimpleXMLRPCServer.SimpleXMLRPCServer( ('', 80) )
server.register_function(test)
server.serve_forever()

-- 
Mit freundlichen Gr??en,
Best Regards,

Robert H?lzl
BALTECH AG

Firmensitz: Lilienthalstrasse 27, D-85399 Hallbergmoos
Registergericht: Amtsgericht M?nchen, HRB 115215
Vorstand: J?rgen R?sch (Vorsitzender), Martina M. Schuster
Aufsichtsratsvorsitzende: Eva Zeising

-------------- next part --------------
A non-text attachment was scrubbed...
Name: robert_hoelzl.vcf
Type: text/x-vcard
Size: 569 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080417/535c5441/attachment-0001.vcf>

From guzarva at windowslive.com  Thu Apr 17 23:22:17 2008
From: guzarva at windowslive.com (guzarva16)
Date: Thu, 17 Apr 2008 21:22:17 -0000
Subject: [Python-Dev]  Security Advisory for unicode repr() bug?
Message-ID: <BAY101-DS1E7D72864ED37CC6D5B21A4E50@phx.gbl>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080417/c006960f/attachment.htm>

From alberto.casado.martin at gmail.com  Tue Apr 22 09:43:02 2008
From: alberto.casado.martin at gmail.com (=?ISO-8859-1?Q?Alberto_Casado_Mart=EDn?=)
Date: Tue, 22 Apr 2008 09:43:02 +0200
Subject: [Python-Dev] python hangs when parsing a bad-formed email
Message-ID: <bfd73c140804220043l4c418c50y2ff7c6da7a13c41a@mail.gmail.com>

Hi all,
First of all, sorry if this isn't the list where I have to post this.
And sorry for my english.

As the subject says, I'm having problems with the attached email, when
I try to get a email object  reading the attached file, the python
process gets hang and gets all cpu.

I have debuged my code to find where it happens, and I found that is
_parsegen method of the FeedParser class. I know that the email format
is wrong but  I don't know why python hangs.

following paste the code showing where hangs.

def _parsegen(self):
        # Create a new message and start by parsing headers.
        self._new_message()
        headers = []
        # Collect the headers, searching for a line that doesn't match the RFC
        # 2822 header or continuation pattern (including an empty line).
        for line in self._input:
            if line is NeedMoreData:
                yield NeedMoreData
                continue
            if not headerRE.match(line):
                # If we saw the RFC defined header/body separator
                # (i.e. newline), just throw it away. Otherwise the line is
                # part of the body so push it back.
                if not NLCRE.match(line):
                    self._input.unreadline(line)
                break
            headers.append(line)
        # Done with the headers, so parse them and figure out what we're
        # supposed to see in the body of the message.
        self._parse_headers(headers)
        # Headers-only parsing is a backwards compatibility hack, which was
        # necessary in the older parser, which could throw errors.  All
        # remaining lines in the input are thrown into the message body.
        if self._headersonly:
            lines = []
            while True:
                line = self._input.readline()
                if line is NeedMoreData:
                    yield NeedMoreData
                    continue
                if line == '':
                    break
                lines.append(line)
            self._cur.set_payload(EMPTYSTRING.join(lines))
            return
        if self._cur.get_content_type() == 'message/delivery-status':
!!!!!!  AT THIS POINT HANGS, AND STRAT TO GET ALL CPU FOR THE PROCESS
            # message/delivery-status contains blocks of headers separated by
            # a blank line.  We'll represent each header block as a separate
            # nested message object, but the processing is a bit different
            # than standard message/* types because there is no body for the
            # nested messages.  A blank line separates the subparts.
  ...
  ...
  ...

I have workaround the problem adding this line in _parse_headers method

def _parse_headers(self, lines):
        # Passed a list of lines that make up the headers for the current msg
        lastheader = ''
        lastvalue = []
        for lineno, line in enumerate(lines):
            # Check for continuation
            if line[0] in ' \t':
                if not lastheader:
                    # The first line of the headers was a continuation.  This
                    # is illegal, so let's note the defect, store the illegal
                    # line, and ignore it for purposes of headers.
                    defect = errors.FirstHeaderLineIsContinuationDefect(line)
                    self._cur.defects.append(defect)
                    continue
                if line.strip()!='': !!!!!!! IF THE CONTINUATION LINE
IS NOT EMPTY ADD THE LINE TO THE HEADER.
                    lastvalue.append(line)
                continue
            if lastheader:
    ...
    ...
    ...


I don't know why it hangs and I'm not sure why with this line works......

I have tried to parse this email in python 2.3.3 SunOs, python 2.3.3 gcc
python 2.5.1 SunOs,gcc, Windows Xp, and linux SUSE 10. And I have
alway the same  result.


bash-3.00$ python
Python 2.5.1 (r251:54863, Feb 28 2008, 07:48:25)
[GCC 3.4.6] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> fp = open('raro.txt')
>>> mail = email.message_from_file(fp)
never return............

I don't know if someone can tell me what is happening....

Best Regards.

Alberto Casado.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: raro.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080422/46c9e000/attachment-0001.txt>

From janssen at parc.com  Tue Apr 22 23:42:28 2008
From: janssen at parc.com (Bill Janssen)
Date: Tue, 22 Apr 2008 14:42:28 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480E2918.7090006@v.loewis.de> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
Message-ID: <08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>

> > When a web browser POSTs data, there is no standard way of communicating
> > which encoding it's using.
> 
> That's just not true. Web browser should and do use the encoding of the
> web page that originally contained the form.

Since the site that receives the POST doesn't necessarily have access
to the Web page that originally contained the form, that's not really
helpful.  However, POSTs can use the MIME type "multipart/form-data"
for non-Latin-1 content, and should.  That contains facilities for
indicating the encoding and other things as well.

Bill

From musiccomposition at gmail.com  Wed Apr 23 00:01:31 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 22 Apr 2008 17:01:31 -0500
Subject: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
In-Reply-To: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
References: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
Message-ID: <1afaf6160804221501j1ca2f634kef0ab356bb2a25aa@mail.gmail.com>

On Wed, Apr 9, 2008 at 8:23 PM, Alvin Delagon <adelagon at gmail.com> wrote:
>
> I'm going to construct an SS7 packet in python using struct.pack(). Here's
> the question, how am I going to pass the packet I wrote in python to my
> module and back? I already asked this question in comp.lang.python but so
> far no responses yet. I hope anyone can point me to the right direction.
> Thanks in advance.

What exactly is your problem?



-- 
Cheers,
Benjamin Peterson

From musiccomposition at gmail.com  Wed Apr 23 00:04:57 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 22 Apr 2008 17:04:57 -0500
Subject: [Python-Dev] SetType=set in types module ?
In-Reply-To: <78e18a710804160608j68a07056y63fcd3a0ebab08ca@mail.gmail.com>
References: <78e18a710804160608j68a07056y63fcd3a0ebab08ca@mail.gmail.com>
Message-ID: <1afaf6160804221504j6a83c84byc58329d66950fc96@mail.gmail.com>

On Wed, Apr 16, 2008 at 8:08 AM, iks hefem <ikshefem at gmail.com> wrote:
> Hi,
>
>  the SetType is not available in the "types" module, so wouldn't it be
>  needed here ? (in 2.6 by example)

Nothing new is currently being added to the types module because we
are trying to decide whether to remove it or not. Please file a bug
report, though, to remind us if we decide to keep it.



-- 
Cheers,
Benjamin Peterson

From janssen at parc.com  Wed Apr 23 00:33:44 2008
From: janssen at parc.com (Bill Janssen)
Date: Tue, 22 Apr 2008 15:33:44 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480E59DC.3010007@egenix.com> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de> <480DBE86.2000507@egenix.com>
	<08Apr22.081453pdt."58696"@synergy1.parc.xerox.com>
	<08Apr22.093328pdt."58696"@synergy1.parc.xerox.com>
	<480E59DC.3010007@egenix.com>
Message-ID: <08Apr22.153354pdt."58696"@synergy1.parc.xerox.com>

> Unless you're using a very broad scope, I don't think that
> you'd need more than a few hundred LSEs for a typical
> application - nothing you'd want to put in the Python stdlib,
> though.

I tend to agree with this (and I'm generally in favor of putting
everything in the standard library!).  For those of us doing
document-processing applications (Martin, it's not just about Web
browsers), this would be a very useful package to have up on PyPI.

Bill

From mike.klaas at gmail.com  Wed Apr 23 01:10:17 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 22 Apr 2008 16:10:17 -0700
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480E5592.3050102@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de>
	<480DBE86.2000507@egenix.com>
	<1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>
	<480E5592.3050102@v.loewis.de>
Message-ID: <F735CD02-3E9F-42E3-B63C-8DC17F8E29E9@gmail.com>


On 22-Apr-08, at 2:16 PM, Martin v. L?wis wrote:
>
>> Any program that needs to examine the contents of
>> documents/feeds/whatever on the web needs to deal with
>> incorrectly-specified encodings
>
> That's not true. Most programs that need to examine the contents of
> a web page don't need to guess the encoding. In most such programs,
> the encoding can be hard-coded if the declared encoding is not
> correct. Most such programs *know* what page they are webscraping,
> or else they couldn't extract the information out of it that they
> want to get at.

I certainly agree that if the target set of documents is small enough  
it is possible to hand-code the encoding.  There are many  
applications, however, that need to examine the content of an  
arbitrary, or at least non-small set of web documents.  To name a few  
such applications:

  - web search engines
  - translation software
  - document/bookmark management systems
  - other kinds of document analysis (market research, seo, etc.)

> As for feeds - can you give examples of incorrectly encoded one
> (I don't ever use feeds, so I honestly don't know whether they
> are typically encoded incorrectly. I've heard they are often XML,
> in which case I strongly doubt they are incorrectly encoded)

I also don't have much experience with feeds.  My statement is based  
on the fact that chardet, the tool that has been cited most in this  
thread, was written specifically for use with the author's feed  
parsing package.

> As for "whatever" - can you give specific examples?

Not that I can substantiate.  Documents & feeds covers a lot of what  
is on the web--I was only trying to make the point that on the web,  
whenever an encoding can be specified, it will be specified  
incorrectly for a significant chunk of exemplars.

>> (which, sadly, is rather common). The
>> set of programs of programs that need this functionality is  
>> probably the
>> same set that needs BeautifulSoup--I think that set is larger than  
>> just
>> browsers <grin>
>
> Again, can you give *specific* examples that are not web browsers?
> Programs needing BeautifulSoup may still not need encoding guessing,
> since they still might be able to hard-code the encoding of the web
> page they want to process.

Indeed, if it is only one site it is pretty easy to work around.  My  
main use of python is processing and analyzing hundreds of millions of  
web documents, so it is pretty easy to see applications (which I have  
listed above).  I think that libraries like Mark Pilgrim's FeedParser  
and BeautifulSoup are possible consumers of guessing as well.

> In any case, I'm very skeptical that a general "guess encoding"
> module would do a meaningful thing when applied to incorrectly
> encoded HTML pages.

Well, it does.  I wish I could easily provide data on how often it is  
necessary over the whole web, but that would be somewhat difficult to  
generate.  I can say that it is much more important to be able to  
parse all the different kinds of encoding _specification_ on the web  
(Content-Type/Content-Encoding/<meta http-equiv tags, etc), and the  
malformed cases of these.

I can also think of good arguments for excluding encoding detection  
for maintenance reasons: is every case of the algorithm guessing wrong  
a bug that needs to be fixed in the stdlib?  That is an unbounded  
commitment.

-Mike

From mike.klaas at gmail.com  Wed Apr 23 01:11:29 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 22 Apr 2008 16:11:29 -0700
Subject: [Python-Dev] socket recv on win32 can be extremly delayed
	(python bug?)
In-Reply-To: <480721A9.1070603@baltech.de>
References: <480721A9.1070603@baltech.de>
Message-ID: <ACC9A379-6C27-48A5-A9ED-5C41472B30C8@gmail.com>

Hi,

This is not a python-specific problem. See
http://en.wikipedia.org/wiki/Nagle's_algorithm

-Mike

On 17-Apr-08, at 3:08 AM, Robert H?lzl wrote:
> hello,
>
> I tried to implement a simple python XMLRPC service on a win32  
> environment (client/server code inserted below).
> The profiler of the client told me, that a simple function call  
> needs about 200ms (even if I run it in a loop, the time needed per  
> call stays the same).
>
> After analysing the problem with etherreal I found out, that the  
> XMLRPC request is transmitted via two TCP packets. One containing  
> the HTTP header and one containting the data. But the acknowledge to  
> the first TCP packet is delayed by 200ms.
>
> I tried around on the server side and found out that if the server  
> reads exactly all bytes transfered in the first TCP frame (via  
> socket.recv()), the next socket.recv(), even if reading only one  
> byte, needs about 200 ms. But if I read one byte less than  
> transfered in the first TCP frame and then reading 2 bytes  
> (socket.recv(2)) there is no delay, although the same total amount  
> of data was read.
>
> After some googling I found the website http://support.microsoft.com/?scid=kb%3Ben-us%3B823764&x=12&y=15 
> , which proposed a workaround (modifing the registryentry for the  
> tcp/ip driver) that did work. But modifing the clients registry  
> settings is no option for us.
>
> Is there anybody who nows how to solve the problem? Or is it even a  
> problem if the python socket implementation?
>
> By the way: I testet Win2000 SP4 and WinXP SP2 with Python 2.3.3 and  
> Python 2.5.1 each.
>
> CLIENT:
> ----------
> import xmlrpclib
> import profile
> server = xmlrpclib.ServerProxy("http://server:80")
> profile.run('server.test(1,2)')
>
> SERVER:
> ----------
> import SimpleXMLRPCServer
> def test(a,b): return a+b
> server = SimpleXMLRPCServer.SimpleXMLRPCServer( ('', 80) )
> server.register_function(test)
> server.serve_forever()
>
> -- 
> Mit freundlichen Gr??en,
> Best Regards,
>
> Robert H?lzl
> BALTECH AG
>
> Firmensitz: Lilienthalstrasse 27, D-85399 Hallbergmoos
> Registergericht: Amtsgericht M?nchen, HRB 115215
> Vorstand: J?rgen R?sch (Vorsitzender), Martina M. Schuster
> Aufsichtsratsvorsitzende: Eva Zeising
>
> <robert_hoelzl.vcf>_______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mike.klaas%40gmail.com


From amauryfa at gmail.com  Wed Apr 23 01:18:30 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 23 Apr 2008 01:18:30 +0200
Subject: [Python-Dev] python hangs when parsing a bad-formed email
In-Reply-To: <bfd73c140804220043l4c418c50y2ff7c6da7a13c41a@mail.gmail.com>
References: <bfd73c140804220043l4c418c50y2ff7c6da7a13c41a@mail.gmail.com>
Message-ID: <e27efe130804221618lf4efc93p685272eeca10796c@mail.gmail.com>

Hello,

Alberto Casado Mart?n wrote:
> Hi all,
>  First of all, sorry if this isn't the list where I have to post this.
>  And sorry for my english.
>
>  As the subject says, I'm having problems with the attached email, when
>  I try to get a email object  reading the attached file, the python
>  process gets hang and gets all cpu.
>
>  I have debuged my code to find where it happens, and I found that is
>  _parsegen method of the FeedParser class. I know that the email format
>  is wrong but  I don't know why python hangs.
>
>  following paste the code showing where hangs.
[snip]
>  bash-3.00$ python
>  Python 2.5.1 (r251:54863, Feb 28 2008, 07:48:25)
>  [GCC 3.4.6] on sunos5
>  Type "help", "copyright", "credits" or "license" for more information.
>  >>> import email
>  >>> fp = open('raro.txt')
>  >>> mail = email.message_from_file(fp)
>  never return............

When you think you found a problem with python, please submit an issue
in the python issue tracker:
    http://bugs.python.org/

In your case, I suspect some regular expression trying to match all
the empty lines of the message, one character at a time.

-- 
Amaury Forgeot d'Arc

From rbp at isnomore.net  Wed Apr 23 01:35:22 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Tue, 22 Apr 2008 20:35:22 -0300
Subject: [Python-Dev] GSoC student introduction and sandbox commit
	privileges request
Message-ID: <20080422233518.GI16282@isnomore.net>

Hi there,

I've just been accepted into this year's Google Summer of Code, to work for
the Python Software Foundation on 2to3. My project is to give 2to3 fixers
the ability to rank how confident they are on each fix, and let users choose
to intervene manually whenever that confidence level is below a certain
threshold. Among other things, this might allow fixers for situations where
the code translation is not always guaranteed to be correct (like % string
formatting, which came up recently in another thread). The full proposal is
at http://isnomore.net/2to3 .

Collin Winter will be my mentor, and I'd like to thank him and Christian
Heimes for all the help they gave me in designing the project. I'd also like
to thank Martin L?wis, for discussing a project with me which ended up not
turning into a proposal, but helped me write the 2to3 one.

Finally, I'd like to request commit privileges to work on a sandbox branch,
during the Summer of Code.

If you have any further questions, please feel free to contact me. I'm
really looking forward to working on this project!

Cheers,


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From tjreedy at udel.edu  Wed Apr 23 01:35:29 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 22 Apr 2008 19:35:29 -0400
Subject: [Python-Dev] python hangs when parsing a bad-formed email
References: <bfd73c140804220043l4c418c50y2ff7c6da7a13c41a@mail.gmail.com>
	<e27efe130804221618lf4efc93p685272eeca10796c@mail.gmail.com>
Message-ID: <fulsnv$q6f$1@ger.gmane.org>


"Amaury Forgeot d'Arc" <amauryfa at gmail.com> wrote in message 
news:e27efe130804221618lf4efc93p685272eeca10796c at mail.gmail.com...


| When you think you found a problem with python, please submit an issue
| in the python issue tracker:
|    http://bugs.python.org/

Or post to comp.lang.python / python mailing list / 
gmane.comp.python.general 




From janssen at parc.com  Wed Apr 23 01:50:40 2008
From: janssen at parc.com (Bill Janssen)
Date: Tue, 22 Apr 2008 16:50:40 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
	<08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>
	<Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu>
Message-ID: <08Apr22.165049pdt."58696"@synergy1.parc.xerox.com>

> Yup, but DrProject (the target application) also serves as a relay and 
> archive for email.  We have no control over the agent used for 
> composition, and AFAIK there's no standard way to include encoding 
> information.

Greg,

Internet-compliant email actually has well-specified mechanisms for
including encoding information; see RFCs 2047 and 2231.  There's no
need to guess; you can just look.

Bill

From brett at python.org  Wed Apr 23 02:02:49 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Apr 2008 17:02:49 -0700
Subject: [Python-Dev] GSoC student introduction and sandbox commit
	privileges request
In-Reply-To: <20080422233518.GI16282@isnomore.net>
References: <20080422233518.GI16282@isnomore.net>
Message-ID: <bbaeab100804221702h7b63a470gfb60e37d209a0d44@mail.gmail.com>

On Tue, Apr 22, 2008 at 4:35 PM, Rodrigo Bernardo Pimentel
<rbp at isnomore.net> wrote:
> Hi there,
>
>  I've just been accepted into this year's Google Summer of Code, to work for
>  the Python Software Foundation on 2to3. My project is to give 2to3 fixers
>  the ability to rank how confident they are on each fix, and let users choose
>  to intervene manually whenever that confidence level is below a certain
>  threshold. Among other things, this might allow fixers for situations where
>  the code translation is not always guaranteed to be correct (like % string
>  formatting, which came up recently in another thread). The full proposal is
>  at http://isnomore.net/2to3 .
>
>  Collin Winter will be my mentor, and I'd like to thank him and Christian
>  Heimes for all the help they gave me in designing the project. I'd also like
>  to thank Martin L?wis, for discussing a project with me which ended up not
>  turning into a proposal, but helped me write the 2to3 one.
>
>  Finally, I'd like to request commit privileges to work on a sandbox branch,
>  during the Summer of Code.
>

Isn't this a chance for bzr to shine? With lib2to3 in the 3.0 bzr
branch, can't Rodrigo and the other students who don't have some funky
requirement just use bzr?

>  If you have any further questions, please feel free to contact me. I'm
>  really looking forward to working on this project!

Thanks for contributing!

-Brett

From rbp at isnomore.net  Wed Apr 23 02:18:05 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Tue, 22 Apr 2008 21:18:05 -0300
Subject: [Python-Dev] GSoC student introduction and sandbox
	commit	privileges request
In-Reply-To: <bbaeab100804221702h7b63a470gfb60e37d209a0d44@mail.gmail.com>
References: <20080422233518.GI16282@isnomore.net>
	<bbaeab100804221702h7b63a470gfb60e37d209a0d44@mail.gmail.com>
Message-ID: <20080423001804.GK16282@isnomore.net>

On Tue, Apr 22 2008 at 09:02:49PM BRT, Brett Cannon <brett at python.org> wrote:
> On Tue, Apr 22, 2008 at 4:35 PM, Rodrigo Bernardo Pimentel
> <rbp at isnomore.net> wrote:

> >  I've just been accepted into this year's Google Summer of Code
(...)
> >  Finally, I'd like to request commit privileges to work on a sandbox
> > branch, during the Summer of Code.

> Isn't this a chance for bzr to shine? With lib2to3 in the 3.0 bzr
> branch, can't Rodrigo and the other students who don't have some funky
> requirement just use bzr?

FWIW, +1 from me, I'm perfectly comfortable with bzr.

> >  If you have any further questions, please feel free to contact me. I'm
> >  really looking forward to working on this project!
> 
> Thanks for contributing!

My pleasure :)

Cheers,


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From lists at cheimes.de  Wed Apr 23 02:35:31 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 23 Apr 2008 02:35:31 +0200
Subject: [Python-Dev] SetType=set in types module ?
In-Reply-To: <1afaf6160804221504j6a83c84byc58329d66950fc96@mail.gmail.com>
References: <78e18a710804160608j68a07056y63fcd3a0ebab08ca@mail.gmail.com>
	<1afaf6160804221504j6a83c84byc58329d66950fc96@mail.gmail.com>
Message-ID: <480E8453.2000005@cheimes.de>

Benjamin Peterson schrieb:
> On Wed, Apr 16, 2008 at 8:08 AM, iks hefem <ikshefem at gmail.com> wrote:
>> Hi,
>>
>>  the SetType is not available in the "types" module, so wouldn't it be
>>  needed here ? (in 2.6 by example)
> 
> Nothing new is currently being added to the types module because we
> are trying to decide whether to remove it or not. Please file a bug
> report, though, to remind us if we decide to keep it.

Eventually the types module will go away or at least be stripped down in
Python 3.0. New types like the set type weren't added to types
deliberately. Please don't file a bug report.

Christian

From lists at cheimes.de  Wed Apr 23 02:38:58 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 23 Apr 2008 02:38:58 +0200
Subject: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
In-Reply-To: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
References: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
Message-ID: <480E8522.9090008@cheimes.de>

Alvin Delagon schrieb:
> I'm going to construct an SS7 packet in python using struct.pack(). Here's
> the question, how am I going to pass the packet I wrote in python to my
> module and back? I already asked this question in comp.lang.python but so
> far no responses yet. I hope anyone can point me to the right direction.
> Thanks in advance.

The Python developer list is meant for the development OF Python, not
WITH Python. Please use the general Python user list to get help.

Christian

From nedds at uchicago.edu  Wed Apr 23 02:42:32 2008
From: nedds at uchicago.edu (Nick Edds)
Date: Tue, 22 Apr 2008 19:42:32 -0500
Subject: [Python-Dev] GSoC Student Introduction
Message-ID: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>

Hello,

My name is Nick Edds. I am going to be working on the 2to3 tool with Collin
Winter as my mentor. More specifically, I will be working on improving the
performance of the 2to3 tool in general, and its use of patterns in
particular.
I would like to request commit privileges to work in a sandbox branch and
although I don't have any familiarity with bzr, I would be comfortable using
it.

Regards,
Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080422/c984b769/attachment.htm>

From collinw at gmail.com  Wed Apr 23 02:56:00 2008
From: collinw at gmail.com (Collin Winter)
Date: Tue, 22 Apr 2008 17:56:00 -0700
Subject: [Python-Dev] GSoC student introduction and sandbox commit
	privileges request
In-Reply-To: <20080423001804.GK16282@isnomore.net>
References: <20080422233518.GI16282@isnomore.net>
	<bbaeab100804221702h7b63a470gfb60e37d209a0d44@mail.gmail.com>
	<20080423001804.GK16282@isnomore.net>
Message-ID: <43aa6ff70804221756k57626854x78bf2c45ccea3653@mail.gmail.com>

On Tue, Apr 22, 2008 at 5:18 PM, Rodrigo Bernardo Pimentel
<rbp at isnomore.net> wrote:
> On Tue, Apr 22 2008 at 09:02:49PM BRT, Brett Cannon <brett at python.org> wrote:
>  > On Tue, Apr 22, 2008 at 4:35 PM, Rodrigo Bernardo Pimentel
>  > <rbp at isnomore.net> wrote:
>
>
> > >  I've just been accepted into this year's Google Summer of Code
>  (...)
>
> > >  Finally, I'd like to request commit privileges to work on a sandbox
>  > > branch, during the Summer of Code.
>
>  > Isn't this a chance for bzr to shine? With lib2to3 in the 3.0 bzr
>  > branch, can't Rodrigo and the other students who don't have some funky
>  > requirement just use bzr?
>
>  FWIW, +1 from me, I'm perfectly comfortable with bzr.

Fine by me; I don't care one way or the other.

Collin

>  > >  If you have any further questions, please feel free to contact me. I'm
>  > >  really looking forward to working on this project!
>  >
>  > Thanks for contributing!
>
>  My pleasure :)
>
>  Cheers,
>
>
>         rbp
>  --
>  Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>
>
>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/collinw%40gmail.com
>

From musiccomposition at gmail.com  Wed Apr 23 04:38:16 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 22 Apr 2008 21:38:16 -0500
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
Message-ID: <1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>

On Tue, Apr 22, 2008 at 7:42 PM, Nick Edds <nedds at uchicago.edu> wrote:
> Hello,
>
> My name is Nick Edds. I am going to be working on the 2to3 tool with Collin
> Winter as my mentor. More specifically, I will be working on improving the
> performance of the 2to3 tool in general, and its use of patterns in
> particular.

>  I would like to request commit privileges to work in a sandbox branch and
> although I don't have any familiarity with bzr, I would be comfortable using
> it.

Luckily, Bazaar is really easy.

Thanks for contributing!



-- 
Cheers,
Benjamin Peterson

From jcea at argo.es  Wed Apr 23 05:09:49 2008
From: jcea at argo.es (Jesus Cea)
Date: Wed, 23 Apr 2008 05:09:49 +0200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
Message-ID: <480EA87D.5040107@argo.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trent Nelson wrote:
| I remember those rampant BSDDB crashes on Windows well.
[...]
| basically, the tests weren't cleaning up their environment in
| the right order, so BSDDB was getting passed completely and
| utterly bogus values.

Next week I will (if nothing goes wrong) publish pybsddb 4.6.4. This
release supports distributed transactions and replication, testsuite is
way faster, and rewritten to be able to launch tests from multiple
threads/processes if you wish, setuptools/pypi support, etc.

I think this release would be appropiate to integrate in Python. I think
most demands are solved and new features are interesting (replication,
distributed transactions, do not crash when closing objects in the wrong
order...). Also, I completed the documentation, with the full supported
API, and ported it to Python 2.6 documentation system. The result:

http://www.jcea.es/programacion/pybsddb.htm#bsddb3-4.6.4
http://www.jcea.es/programacion/pybsddb_doc/preview/

I'm very interested in integrating this release in Python 2.6 for the
new features, the full documentation, and to get feedback from Buildbot
and python-dev community. Also, I would like to avoid to integrate
pybsddb late in the python 2.6 release cycle; I hope to be away of my
computer in August! :).

I'm a bit nervous about syncing, because I have the feeling that
python-dev is committing changes to python private branch of pybsddb. I
would rather prefer patches send to me and integrate "canonical" pybsddb
releases in Python frequently.

Somebody suggested to post patches in the tracker, but I think this is
not going to work. The diff from current python bsddb and the official
version is so huge that nobody could follow it. A more sensible
approach, I think, is to "diff" current python pybsddb against the
version I used as my root (January?), integrate the changes in current
"canonical" pybsddb and, then, drop the entire updated package into python.

Then, commits to python pybsddb should be avoided; patches should be
send to me.

I think this is the only way when integrating a project outside python
SVN. Suggestions?.

PS: I can't comment on Win64. It is an alien world to me :).

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
~                               _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSA6oeJlgi5GaxT1NAQItswP+KR15vZWbnYZ23WQHoUozVOWvf+ghG2Q8
acVhCwJajzvxOEfozRMZRmQkPUBmWga1zbHjkHt5c196vku7+X0bDc7aO4T2jRHx
00PbPLGnYth972elTVFfSWpZVNkX/9A4EbtTHVCav105nW+u1/Kod/rY5fzgKcTn
SxYkmk4Ax7U=
=98uc
-----END PGP SIGNATURE-----

From brett at python.org  Wed Apr 23 05:45:22 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Apr 2008 20:45:22 -0700
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
Message-ID: <bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>

On Tue, Apr 22, 2008 at 7:38 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
>
> On Tue, Apr 22, 2008 at 7:42 PM, Nick Edds <nedds at uchicago.edu> wrote:
>  > Hello,
>  >
>  > My name is Nick Edds. I am going to be working on the 2to3 tool with Collin
>  > Winter as my mentor. More specifically, I will be working on improving the
>  > performance of the 2to3 tool in general, and its use of patterns in
>  > particular.
>
>  >  I would like to request commit privileges to work in a sandbox branch and
>  > although I don't have any familiarity with bzr, I would be comfortable using
>  > it.
>
>  Luckily, Bazaar is really easy.
>

See http://python.org/dev/bazaar/ for info. And if you have any other
issues feel free to ask, Nick.

-Brett

From stephen at xemacs.org  Wed Apr 23 06:44:20 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Apr 2008 13:44:20 +0900
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <08Apr22.165049pdt."58696"@synergy1.parc.xerox.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
	<08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>
	<Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu>
	<08Apr22.165049pdt."58696"@synergy1.parc.xerox.com>
Message-ID: <87ej8xp4d7.fsf@uwakimon.sk.tsukuba.ac.jp>

Bill Janssen writes:

 > Internet-compliant email actually has well-specified mechanisms for
 > including encoding information; see RFCs 2047 and 2231.  There's no
 > need to guess; you can just look.

You must be very special to get only compliant email.

About half my colleagues use RFC 2047 to encode Japanese file names in
MIME attachments (a MUST NOT behavior according to RFC 2047), and a
significant fraction of the rest end up with binary Shift JIS or EUC
or MacRoman in there.

And those are just the most widespread violations I can think of off
the top of my head.

Not to mention that I find this:

    =?X-UNKNOWN?Q?Martin_v=2E_L=F6wis?= <martin at v.loewis.de>,

in the header I got from you.  (I'm not ragging on you, I get Martin's
name wrong a significant portion of the time myself. :-( )

From stephen at xemacs.org  Wed Apr 23 06:56:47 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Apr 2008 13:56:47 +0900
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480E5592.3050102@v.loewis.de>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de> <480DBE86.2000507@egenix.com>
	<1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>
	<480E5592.3050102@v.loewis.de>
Message-ID: <87d4ohp3sg.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > In any case, I'm very skeptical that a general "guess encoding"
 > module would do a meaningful thing when applied to incorrectly
 > encoded HTML pages.

That depends on whether you can get meaningful information about the
language from the fact that you're looking at the page.  In the
browser context, for one, 99.44% of users are monolingual, so you only
have to distinguish among the encodings for their language.  In this
context a two stage process of determining a category of encoding (eg,
ISO 8859, ISO 2022 7-bit, ISO 2022 8-bit multibyte, UTF-8, etc), and
then picking an encoding from the category according to a
user-specified configuration has served Emacs/MULE users very well for
about 20 years.

It does *not* work in a context where multiple encodings from the same
category are in use (eg, the email folder of a Polish Gastarbeiter in
Berlin).

Nonetheless it is pretty useful for user agents like mail clients, web
browsers, and editors.

From stephen at xemacs.org  Wed Apr 23 06:59:50 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Apr 2008 13:59:50 +0900
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <ca471dc20804211038n667fdaf5rac62b0a73eebf0ce@mail.gmail.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<480CC6D7.2020404@cheimes.de> <fuiimb$igj$2@ger.gmane.org>
	<ca471dc20804211038n667fdaf5rac62b0a73eebf0ce@mail.gmail.com>
Message-ID: <87bq41p3nd.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > To the contrary, an encoding-guessing module is often needed, and
 > guessing can be done with a pretty high success rate. Other Unicode
 > libraries (e.g. ICU) contain guessing modules. I suppose the API could
 > return two values: the guessed encoding and a confidence indicator.
 > Note that the locale settings might figure in the guess.

Not locale settings, but user configuration.  A Bayesian detector
(CodeBayes? hi, Skip!) might be a good way to go for servers, while a
simple language preference might really up the probability for user
agents.


From martin at v.loewis.de  Wed Apr 23 06:52:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 06:52:06 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
	<08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>
	<Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu>
Message-ID: <480EC076.4090207@v.loewis.de>

> Yup, but DrProject (the target application) also serves as a relay and
> archive for email.  We have no control over the agent used for
> composition, and AFAIK there's no standard way to include encoding
> information.

That's not at all the case. MIME defines that in full detail, since
1993.

Regards,
Martin

From martin at v.loewis.de  Wed Apr 23 07:04:54 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 07:04:54 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <F735CD02-3E9F-42E3-B63C-8DC17F8E29E9@gmail.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de>
	<480DBE86.2000507@egenix.com>
	<1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com>
	<480E5592.3050102@v.loewis.de>
	<F735CD02-3E9F-42E3-B63C-8DC17F8E29E9@gmail.com>
Message-ID: <480EC376.8070406@v.loewis.de>

> I certainly agree that if the target set of documents is small enough it
> is possible to hand-code the encoding.  There are many applications,
> however, that need to examine the content of an arbitrary, or at least
> non-small set of web documents.  To name a few such applications:
> 
>  - web search engines
>  - translation software

I'll question whether these are "many" programs. Web search engines
and translation software have many more challenges to master, and
they are fairly special-cased, so I would expect they need to find
their own answer to character set detection, anyway (see Bill Janssen's
answer on machine translation, also).

>  - document/bookmark management systems
>  - other kinds of document analysis (market research, seo, etc.)

Not sure what specifically you have in mind, however, I expect that
these also have their own challenges. For example, I would expect
that MS-Word documents are frequent. You don't need character set
detection there (Word is all Unicode), but you need an API to look
into the structure of .doc files.

> Not that I can substantiate.  Documents & feeds covers a lot of what is
> on the web--I was only trying to make the point that on the web,
> whenever an encoding can be specified, it will be specified incorrectly
> for a significant chunk of exemplars.

I firmly believe this assumption is false. If the encoding comes out of
software (which it often does), it will be correct most of the time.
It's incorrect only if the content editor has to type it.

> Indeed, if it is only one site it is pretty easy to work around.  My
> main use of python is processing and analyzing hundreds of millions of
> web documents, so it is pretty easy to see applications (which I have
> listed above).

Ok. What advantage would you (or somebody working on a similar project)
gain if chardet was part of the standard library? What if it was not
chardet, but some other algorithm?

> I can also think of good arguments for excluding encoding detection for
> maintenance reasons: is every case of the algorithm guessing wrong a bug
> that needs to be fixed in the stdlib?  That is an unbounded commitment.

Indeed, that's what I meant with my initial remark. People will expect
that it works correctly - both with the consequence of unknowingly
proceeding with the incorrect response, and then complaining when they
find out that it did produce an incorrect answer.

For chardet specifically, my usual standard-library remark applies:
it can't become part of the standard library unless the original
author contributes it, anyway. I would then hope that he or a group
of people would volunteer to maintain it, with the threat of removing
it from the stdlib again if these volunteers go away and too many
problems show up.

Regards,
Martin


From tjreedy at udel.edu  Wed Apr 23 07:26:22 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 23 Apr 2008 01:26:22 -0400
Subject: [Python-Dev] Encoding detection in the standard library?
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de><480DBE86.2000507@egenix.com><1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com><480E5592.3050102@v.loewis.de><F735CD02-3E9F-42E3-B63C-8DC17F8E29E9@gmail.com>
	<480EC376.8070406@v.loewis.de>
Message-ID: <fumh9s$68l$1@ger.gmane.org>


""Martin v. L?wis"" <martin at v.loewis.de> wrote in message 
news:480EC376.8070406 at v.loewis.de...
|> I certainly agree that if the target set of documents is small enough it
|
| Ok. What advantage would you (or somebody working on a similar project)
| gain if chardet was part of the standard library? What if it was not
| chardet, but some other algorithm?

It seems to me that since there is not a 'correct' algorithm but only 
competing heuristics, encoding detection modules should be made available 
via PyPI and only be considered for stdlib after a best of breed emerges 
with community support. 




From martin at v.loewis.de  Wed Apr 23 07:49:13 2008
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 07:49:13 +0200
Subject: [Python-Dev] socket recv on win32 can be extremly delayed
 (python bug?)
In-Reply-To: <480721A9.1070603@baltech.de>
References: <480721A9.1070603@baltech.de>
Message-ID: <480ECDD9.3070101@v.loewis.de>

> Is there anybody who nows how to solve the problem?

If it's really the problem described in the MSKB article, the article
also suggests a solution: use non-blocking sockets.

Regards,
Martin

From martin at v.loewis.de  Wed Apr 23 08:03:13 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 08:03:13 +0200
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
Message-ID: <480ED121.7090005@v.loewis.de>

> See http://python.org/dev/bazaar/ for info. And if you have any other
> issues feel free to ask, Nick.

I certainly can't speak for the respective mentors, but I feel that
"use bazaar" really isn't the right answer to "can I get commit access?"

One motivation for GSoC is also community bonding, and having the
mentor (but not *only* the mentor) comment on the proposed changes,
and monitor the progress of the project. That the development branch
sits on the student's laptop doesn't really help in that process.

Instead, the student would have to push the branch somewhere to
a web-visible location. Now I question whether it's the student's
obligation to find a server himself, or whether the mentoring org
should provide the infrastructure (or, failing that, Google (*)).

So I think an answer to the question above involving bazaar might
be "yes, but please don't commit to subversion, but only to the
bazaar repository".

Regards,
Martin

(*) FWIW, Google does provide the infrastructure; students are
encouraged (required?) to commit their work to code.google.com.

From martin at v.loewis.de  Wed Apr 23 08:12:45 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 08:12:45 +0200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480EA87D.5040107@argo.es>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>	<480CABFB.2080302@argo.es>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
	<480EA87D.5040107@argo.es>
Message-ID: <480ED35D.1040403@v.loewis.de>

> I'm a bit nervous about syncing, because I have the feeling that
> python-dev is committing changes to python private branch of pybsddb. I
> would rather prefer patches send to me and integrate "canonical" pybsddb
> releases in Python frequently.

My understanding was that the pybsddb copy in Python *is* the official
code base. I think it's unfortunate that the pybsddb project was
revived, even though it was already dead.

Perhaps we should remove bsddb from Python again, and refer people to
pybsddb instead?

> Somebody suggested to post patches in the tracker, but I think this is
> not going to work. The diff from current python bsddb and the official
> version is so huge that nobody could follow it. A more sensible
> approach, I think, is to "diff" current python pybsddb against the
> version I used as my root (January?), integrate the changes in current
> "canonical" pybsddb and, then, drop the entire updated package into python.

-1. Who is going to do the 3k porting?

> I think this is the only way when integrating a project outside python
> SVN. Suggestions?.

The usual solution is to not integrate then, at all. Python doesn't
really ship with any libraries that also have an active life outside
of Python.

Regards,
Martin

From hrvoje.niksic at avl.com  Wed Apr 23 09:00:57 2008
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Wed, 23 Apr 2008 09:00:57 +0200
Subject: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
In-Reply-To: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
References: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
Message-ID: <1208934057.11332.1.camel@localhost>

On Thu, 2008-04-10 at 09:23 +0800, Alvin Delagon wrote:
> I'm currently writing a simple python SCTP module in C. So far it
> works sending and receiving strings from it.
[...]
> I'm going to construct an SS7 packet in python using struct.pack(). 
> Here's the question, how am I going to pass the packet I wrote in 
> python to my module and back? I already asked this question in 
> comp.lang.python but so far no responses yet.

Have you tried posting to the C API mailing list?

http://mail.python.org/mailman/listinfo/capi-sig



From p.f.moore at gmail.com  Wed Apr 23 10:34:44 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 23 Apr 2008 09:34:44 +0100
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <480ED121.7090005@v.loewis.de>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
	<480ED121.7090005@v.loewis.de>
Message-ID: <79990c6b0804230134v7238238dh57ad85e3852c752a@mail.gmail.com>

On 23/04/2008, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > See http://python.org/dev/bazaar/ for info. And if you have any other
> > issues feel free to ask, Nick.
>
> I certainly can't speak for the respective mentors, but I feel that
> "use bazaar" really isn't the right answer to "can I get commit access?"
[...]
> So I think an answer to the question above involving bazaar might
> be "yes, but please don't commit to subversion, but only to the
> bazaar repository".

Agreed. Can I also point out that Bazaar isn't an official repository
for Python (yet). I don't want to open up the DVCS discussion again,
but I have recently finished looking at both Bazaar and Mercurial for
my personal use (including managing Python contributions) and decided
on Mercurial. A significant part of the decision was the fact that in
certain configurations, Bazaar is really, really slow.

It feels a little like Bazaar is becoming the "de facto" DVCS solution
for Python, and I'd like to avoid that happening until a proper
evaluation has been done.

Paul.

PS If the fact that Bazaar repos are set up is an important factor
here, I'd be willing to set up the equivalent Mercurial repos -
although I would need access to the relevant boxes and possibly some
admin assistance.

From mal at egenix.com  Wed Apr 23 12:01:18 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 23 Apr 2008 12:01:18 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <fumh9s$68l$1@ger.gmane.org>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>	<18444.50265.618909.751390@montanaro-dyndns-org.local>	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>	<480D079A.2020904@v.loewis.de><480DBE86.2000507@egenix.com><1A9B6931-5D55-4442-BB50-0B888AA98086@gmail.com><480E5592.3050102@v.loewis.de><F735CD02-3E9F-42E3-B63C-8DC17F8E29E9@gmail.com>	<480EC376.8070406@v.loewis.de>
	<fumh9s$68l$1@ger.gmane.org>
Message-ID: <480F08EE.1050207@egenix.com>

On 2008-04-23 07:26, Terry Reedy wrote:
> ""Martin v. L?wis"" <martin at v.loewis.de> wrote in message 
> news:480EC376.8070406 at v.loewis.de...
> |> I certainly agree that if the target set of documents is small enough it
> |
> | Ok. What advantage would you (or somebody working on a similar project)
> | gain if chardet was part of the standard library? What if it was not
> | chardet, but some other algorithm?
> 
> It seems to me that since there is not a 'correct' algorithm but only 
> competing heuristics, encoding detection modules should be made available 
> via PyPI and only be considered for stdlib after a best of breed emerges 
> with community support. 

+1

Though in practice, determining the "best of breed" often becomes a
problem (see e.g. the JSON implementation discussion).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 23 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ggpolo at gmail.com  Wed Apr 23 12:24:12 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 23 Apr 2008 07:24:12 -0300
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <480ED121.7090005@v.loewis.de>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
	<480ED121.7090005@v.loewis.de>
Message-ID: <ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>

2008/4/23, "Martin v. L?wis" <martin at v.loewis.de>:
> > See http://python.org/dev/bazaar/ for info. And if you have any other
>  > issues feel free to ask, Nick.
>
>
> I certainly can't speak for the respective mentors, but I feel that
>  "use bazaar" really isn't the right answer to "can I get commit access?"
>
>  One motivation for GSoC is also community bonding, and having the
>  mentor (but not *only* the mentor) comment on the proposed changes,
>  and monitor the progress of the project. That the development branch
>  sits on the student's laptop doesn't really help in that process.
>
>  Instead, the student would have to push the branch somewhere to
>  a web-visible location. Now I question whether it's the student's
>  obligation to find a server himself, or whether the mentoring org
>  should provide the infrastructure (or, failing that, Google (*)).
>
>  So I think an answer to the question above involving bazaar might
>  be "yes, but please don't commit to subversion, but only to the
>  bazaar repository".

Hi Martin,

What is the point on having a branch in the svn repo if you won't be
able to commit ? Maybe I misunderstood what you said, so maybe could
you clarify that answer ?

>
>  Regards,
>  Martin
>
>  (*) FWIW, Google does provide the infrastructure; students are
>  encouraged (required?) to commit their work to code.google.com.

It is required to submit the "final and complete" work done during
GSoC to a project under code.google.com. That project will be
automatically created after gsoc ends. But yeh, we could create
another project there to use as a repo for the summer.

And thank you, James Tauber, effbot and everyone else that accepted me
as a student this year too. I guess I should create a new thread to
introduce myself and my project =)

Thanks,

>
> _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>


-- 
-- Guilherme H. Polo Goncalves

From ndbecker2 at gmail.com  Wed Apr 23 12:48:03 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 23 Apr 2008 06:48:03 -0400
Subject: [Python-Dev] pydoc works with eggs? (python-2.5.1)
References: <fukhg4$ke6$1@ger.gmane.org>
Message-ID: <fun453$un9$1@ger.gmane.org>

Neal Becker wrote:

> pydoc blew up when I tried to view doc for pytools module, which is an
> egg:
> 
> pydoc -p 8082
> pydoc server ready at http://localhost:8082/
> ----------------------------------------
> Exception happened during processing of request from ('127.0.0.1', 52915)
> Traceback (most recent call last):
>   File "/usr/lib64/python2.5/SocketServer.py", line 222, in handle_request
>     self.process_request(request, client_address)
>   File "/usr/lib64/python2.5/SocketServer.py", line 241, in
>   process_request
>     self.finish_request(request, client_address)
>   File "/usr/lib64/python2.5/SocketServer.py", line 254, in finish_request
>     self.RequestHandlerClass(request, client_address, self)
>   File "/usr/lib64/python2.5/SocketServer.py", line 522, in __init__
>     self.handle()
>   File "/usr/lib64/python2.5/BaseHTTPServer.py", line 316, in handle
>     self.handle_one_request()
>   File "/usr/lib64/python2.5/BaseHTTPServer.py", line 310, in
>   handle_one_request
>     method()
>   File "/usr/lib64/python2.5/pydoc.py", line 1924, in do_GET
>     self.send_document(describe(obj), html.document(obj, path))
>   File "/usr/lib64/python2.5/pydoc.py", line 321, in document
>     if inspect.ismodule(object): return self.docmodule(*args)
>   File "/usr/lib64/python2.5/pydoc.py", line 672, in docmodule
>     contents.append(self.document(value, key, name, fdict, cdict))
>   File "/usr/lib64/python2.5/pydoc.py", line 322, in document
>     if inspect.isclass(object): return self.docclass(*args)
>   File "/usr/lib64/python2.5/pydoc.py", line 807, in docclass
>     lambda t: t[1] == 'method')
>   File "/usr/lib64/python2.5/pydoc.py", line 735, in spill
>     funcs, classes, mdict, object))
>   File "/usr/lib64/python2.5/pydoc.py", line 323, in document
>     if inspect.isroutine(object): return self.docroutine(*args)
>   File "/usr/lib64/python2.5/pydoc.py", line 891, in docroutine
>     getdoc(object), self.preformat, funcs, classes, methods)
>   File "/usr/lib64/python2.5/pydoc.py", line 79, in getdoc
>     result = inspect.getdoc(object) or inspect.getcomments(object)
>   File "/usr/lib64/python2.5/inspect.py", line 521, in getcomments
>     lines, lnum = findsource(object)
>   File "/usr/lib64/python2.5/inspect.py", line 510, in findsource
>     if pat.match(lines[lnum]): break
> IndexError: list index out of range
> ----------------------------------------
> 
I see that installing the egg unzipped fixes this.  It looks to me that
pydoc doesn't work with zipped eggs.


From lists at cheimes.de  Wed Apr 23 12:56:44 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 23 Apr 2008 12:56:44 +0200
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <480ED121.7090005@v.loewis.de>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
	<480ED121.7090005@v.loewis.de>
Message-ID: <480F15EC.10905@cheimes.de>

Martin v. L?wis schrieb:
> I certainly can't speak for the respective mentors, but I feel that
> "use bazaar" really isn't the right answer to "can I get commit access?"
> 
> One motivation for GSoC is also community bonding, and having the
> mentor (but not *only* the mentor) comment on the proposed changes,
> and monitor the progress of the project. That the development branch
> sits on the student's laptop doesn't really help in that process.

Students have to sign the contributor agreement anyway or we can't use
their work. I don't see a problem with giving them svn commit privileges
as long as they restrict themselves to their sandbox.

> (*) FWIW, Google does provide the infrastructure; students are
> encouraged (required?) to commit their work to code.google.com.

People doesn't require the students to use their infrastructure. Google
just wants three evaluations from the mentors (after the community
bounding phase, mid term and at the end). Google also requires the
student to send in a tarball with their work - patches, docs, tests and
so on. The tarball is just required for legal reasons. Leslie once
explained it's a social and legal hack to make the contract work and the
lawyers happy.

Christian

From musiccomposition at gmail.com  Wed Apr 23 13:55:34 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 23 Apr 2008 06:55:34 -0500
Subject: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
In-Reply-To: <1208934057.11332.1.camel@localhost>
References: <7a01f6c00804091823x74e29f75jaf4dd3ad53481b1a@mail.gmail.com>
	<1208934057.11332.1.camel@localhost>
Message-ID: <1afaf6160804230455r5a9e484dg8da870a444fefd81@mail.gmail.com>

He emailed me back saying that he found the solution to his problem.

-- 
Cheers,
Benjamin Peterson

From ncoghlan at gmail.com  Wed Apr 23 15:41:02 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Apr 2008 23:41:02 +1000
Subject: [Python-Dev] Externally developed modules (was Re:  BSDDB3)
In-Reply-To: <480ED35D.1040403@v.loewis.de>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>	<480CABFB.2080302@argo.es>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>	<480EA87D.5040107@argo.es>
	<480ED35D.1040403@v.loewis.de>
Message-ID: <480F3C6E.8040103@gmail.com>

Martin v. L?wis wrote:
>> I think this is the only way when integrating a project outside python
>> SVN. Suggestions?.
> 
> The usual solution is to not integrate then, at all. Python doesn't
> really ship with any libraries that also have an active life outside
> of Python.

Not entirely true - we have the ones listed in PEP 360. However, I think 
of those only Optik (aka optparse) is currently seeing active external 
updates.

Of course, we also have a big warning at the top of the PEP saying 
*Never Ever Do This Again*.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Wed Apr 23 16:03:43 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 23 Apr 2008 10:03:43 -0400
Subject: [Python-Dev] pydoc works with eggs? (python-2.5.1)
In-Reply-To: <fun453$un9$1@ger.gmane.org>
References: <fukhg4$ke6$1@ger.gmane.org>
 <fun453$un9$1@ger.gmane.org>
Message-ID: <20080423140329.97F503A40AC@sparrow.telecommunity.com>

At 06:48 AM 4/23/2008 -0400, Neal Becker wrote:
>Neal Becker wrote:
>
> > pydoc blew up when I tried to view doc for pytools module, which is an
> > egg:
> >
> > pydoc -p 8082
> > pydoc server ready at http://localhost:8082/
> > ----------------------------------------
>...
> >
>I see that installing the egg unzipped fixes this.  It looks to me that
>pydoc doesn't work with zipped eggs.

What's odd about this is that it *did* at one time.  Or at least 
help() did.  The changes I made in 2.5 were mainly so that 
help(package) would work on zipped eggs.

 From the traceback, it looks like the issue is that it's trying to 
parse comments out of the source for an object with no docstring.  I 
didn't know it could do that, so I never tried testing it.


From janssen at parc.com  Wed Apr 23 18:16:55 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 23 Apr 2008 09:16:55 PDT
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <480E2918.7090006@v.loewis.de> 
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
Message-ID: <08Apr23.091700pdt."58696"@synergy1.parc.xerox.com>

martin at v.loewis.de writes:
> > When a web browser POSTs data, there is no standard way of communicating
> > which encoding it's using.
> 
> That's just not true. Web browser should and do use the encoding of the
> web page that originally contained the form.

I wonder if the discussion is confusing two different things.  Take a
look at
<http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.13.4>.

There are two prescribed ways of sending form data:
application/x-www-form-urlencoded, which can only be used with ASCII
data, and multipart/form-data.  ``The content type
"multipart/form-data" should be used for submitting forms that contain
files, non-ASCII data, and binary data.''

It's true that the page containing the form may specify which of these
two forms to use, but the character encodings are determined by the
choice.

> For web forms, I always encode the pages in UTF-8, and that always
> works.

Should work, if you use the "multipart/form-data" format.

Bill


From brett at python.org  Wed Apr 23 19:52:20 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Apr 2008 10:52:20 -0700
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480ED35D.1040403@v.loewis.de>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>
	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>
	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>
	<20080227141322.GA5871@amk-desktop.matrixgroup.net>
	<47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
	<480EA87D.5040107@argo.es> <480ED35D.1040403@v.loewis.de>
Message-ID: <bbaeab100804231052h19fda778g5834c701fb840449@mail.gmail.com>

On Tue, Apr 22, 2008 at 11:12 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I'm a bit nervous about syncing, because I have the feeling that
>  > python-dev is committing changes to python private branch of pybsddb. I
>  > would rather prefer patches send to me and integrate "canonical" pybsddb
>  > releases in Python frequently.
>
>  My understanding was that the pybsddb copy in Python *is* the official
>  code base. I think it's unfortunate that the pybsddb project was
>  revived, even though it was already dead.
>
>  Perhaps we should remove bsddb from Python again, and refer people to
>  pybsddb instead?
>

+1 from me! The amount of issues the bsddb module has caused over the
years suggests to me it would do better as an external project again.

-Brett

From rbp at isnomore.net  Wed Apr 23 20:02:40 2008
From: rbp at isnomore.net (Rodrigo Bernardo Pimentel)
Date: Wed, 23 Apr 2008 15:02:40 -0300
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <480F15EC.10905@cheimes.de>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
	<480ED121.7090005@v.loewis.de> <480F15EC.10905@cheimes.de>
Message-ID: <20080423180239.GU16282@isnomore.net>

On Wed, Apr 23 2008 at 07:56:44AM BRT, Christian Heimes <lists at cheimes.de> wrote:
> Martin v. L?wis schrieb:
> > I certainly can't speak for the respective mentors, but I feel that
> > "use bazaar" really isn't the right answer to "can I get commit access?"
> > 
> > One motivation for GSoC is also community bonding, and having the
> > mentor (but not *only* the mentor) comment on the proposed changes,
> > and monitor the progress of the project. That the development branch
> > sits on the student's laptop doesn't really help in that process.

Agreed. Using bazaar, at least in this context, involves pushing changes
regularly to someplace visible, where the mentor and other developers may
see and comment.

> Students have to sign the contributor agreement anyway or we can't use
> their work. I don't see a problem with giving them svn commit privileges
> as long as they restrict themselves to their sandbox.

With that, http://python.org/dev/bazaar/ says "anybody with write access to
the Subversion repository can push their own branches to python.org", so
GSoC students could use bzr and push their changes to python.org regularly,
for mentor (and, more generally, core developers) comments.

Cheers,


        rbp
-- 
Rodrigo Bernardo Pimentel <rbp at isnomore.net> | GPG: <0x0DB14978>

From greg at krypto.org  Wed Apr 23 22:17:19 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 23 Apr 2008 13:17:19 -0700
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480EA87D.5040107@argo.es>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>
	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>
	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>
	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>
	<20080227141322.GA5871@amk-desktop.matrixgroup.net>
	<47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
	<480EA87D.5040107@argo.es>
Message-ID: <52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com>

On Tue, Apr 22, 2008 at 8:09 PM, Jesus Cea <jcea at argo.es> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Trent Nelson wrote:
> | I remember those rampant BSDDB crashes on Windows well.
> [...]
> | basically, the tests weren't cleaning up their environment in
> | the right order, so BSDDB was getting passed completely and
> | utterly bogus values.
>
> Next week I will (if nothing goes wrong) publish pybsddb 4.6.4. This
> release supports distributed transactions and replication, testsuite is
> way faster, and rewritten to be able to launch tests from multiple
> threads/processes if you wish, setuptools/pypi support, etc.
>
> I think this release would be appropiate to integrate in Python. I think
> most demands are solved and new features are interesting (replication,
> distributed transactions, do not crash when closing objects in the wrong
> order...). Also, I completed the documentation, with the full supported
> API, and ported it to Python 2.6 documentation system. The result:
>
> http://www.jcea.es/programacion/pybsddb.htm#bsddb3-4.6.4
> http://www.jcea.es/programacion/pybsddb_doc/preview/
>
> I'm very interested in integrating this release in Python 2.6 for the
> new features, the full documentation, and to get feedback from Buildbot
> and python-dev community. Also, I would like to avoid to integrate
> pybsddb late in the python 2.6 release cycle; I hope to be away of my
> computer in August! :).
>
> I'm a bit nervous about syncing, because I have the feeling that
> python-dev is committing changes to python private branch of pybsddb. I
> would rather prefer patches send to me and integrate "canonical" pybsddb
> releases in Python frequently.
>
> Somebody suggested to post patches in the tracker, but I think this is
> not going to work. The diff from current python bsddb and the official
> version is so huge that nobody could follow it. A more sensible
> approach, I think, is to "diff" current python pybsddb against the
> version I used as my root (January?), integrate the changes in current
> "canonical" pybsddb and, then, drop the entire updated package into
> python.
>

Agreed.  I "forked" jcea's version found in his svn repository from python
trunk at r60209.

The only change to any Modules/*bsddb*  files made to python trunk since
then was me manually applying one bugfix that jcea had made in his
repository.  My goal all along had been to merge his changes back into
python trunk until he had commit access.  Obviously I haven't had time to do
that.

The changes to Lib/bsddb/* since then have been test suite updates which I
think jcea should go through and selectively merge as appropriate into his
repository (he has done a lot of test suite cleanup). ( svn://
svn.argo.es/jcea/pybsddb/trunk )

After that, merging his changes into trunk will be relatively easy and I
think we should give jcea commit access and let him do it and henceforth use
python svn as the official repository for the pybsddb code and
documentation.

I do not see this as a daunting task assuming jcea is willing to be
volunteered to do the merge and to handle any py3k merge issues that come
up.


Alternatively: (removal of bsddb from the standard library)

If people want to go the other route a remove bsddb from Python all together
letting it thrive as a standalone addon module, what will this mean to the
old shelve and dbhash standard library that use bsddb?  Someone should write
a sqlite backend for shelve.  Either way it should not be removed from 2.6
as it needs to be marked at deprecated for one release round only to be
taken out in 2.7 and that does leave open the question of should it exist in
3.0 at all or just start out deprecated there?


-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080423/709002de/attachment.htm>

From martin at v.loewis.de  Wed Apr 23 22:35:52 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 22:35:52 +0200
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>	
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>	
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>	
	<480ED121.7090005@v.loewis.de>
	<ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>
Message-ID: <480F9DA8.4030103@v.loewis.de>

> What is the point on having a branch in the svn repo if you won't be
> able to commit ? Maybe I misunderstood what you said, so maybe could
> you clarify that answer ?

What I meant is this: you need commit privileges regardless of whether
you are going to use bazaar or subversion (i.e. even for bazaar, you
still need commit privileges). So if mentors favor usage of bazaar
over subversion, they still need to arrange their students to get
commit privileges, and then ask them not to use these privileges for
subversion, but only for bazaar (because we only have a single set
of credentials that we manage).

HTH,
Martin

From martin at v.loewis.de  Wed Apr 23 22:38:17 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 22:38:17 +0200
Subject: [Python-Dev] Externally developed modules (was Re:  BSDDB3)
In-Reply-To: <480F3C6E.8040103@gmail.com>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<fpuf2u$6se$1@ger.gmane.org>	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>	<47C32D2D.1030206@free.fr>	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>	<47C4813F.2080403@v.loewis.de>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>	<480CABFB.2080302@argo.es>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>	<480EA87D.5040107@argo.es>
	<480ED35D.1040403@v.loewis.de> <480F3C6E.8040103@gmail.com>
Message-ID: <480F9E39.60600@v.loewis.de>

>> The usual solution is to not integrate then, at all. Python doesn't
>>  really ship with any libraries that also have an active life
>> outside of Python.
> 
> Not entirely true - we have the ones listed in PEP 360. However, I
> think of those only Optik (aka optparse) is currently seeing active
> external updates.

Hence my classification "doesn't really", which I meant to be vague.

> Of course, we also have a big warning at the top of the PEP saying 
> *Never Ever Do This Again*.

I think that's what Guido once said.

Regards,
Martin

From martin at v.loewis.de  Wed Apr 23 22:43:29 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 22:43:29 +0200
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <08Apr23.091700pdt."58696"@synergy1.parc.xerox.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
	<08Apr23.091700pdt."58696"@synergy1.parc.xerox.com>
Message-ID: <480F9F71.7050103@v.loewis.de>

>> For web forms, I always encode the pages in UTF-8, and that always
>> works.
> 
> Should work, if you use the "multipart/form-data" format.

Right - I was implicitly assuming that.

Regards,
Martin

From ggpolo at gmail.com  Wed Apr 23 23:07:36 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 23 Apr 2008 18:07:36 -0300
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <480F9DA8.4030103@v.loewis.de>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>
	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>
	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>
	<480ED121.7090005@v.loewis.de>
	<ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>
	<480F9DA8.4030103@v.loewis.de>
Message-ID: <ac2200130804231407l3671018bm4d21ccd364a60482@mail.gmail.com>

2008/4/23 "Martin v. L?wis" <martin at v.loewis.de>:
> > What is the point on having a branch in the svn repo if you won't be
>  > able to commit ? Maybe I misunderstood what you said, so maybe could
>  > you clarify that answer ?
>
>  What I meant is this: you need commit privileges regardless of whether
>  you are going to use bazaar or subversion (i.e. even for bazaar, you
>  still need commit privileges). So if mentors favor usage of bazaar
>  over subversion, they still need to arrange their students to get
>  commit privileges, and then ask them not to use these privileges for
>  subversion, but only for bazaar (because we only have a single set
>  of credentials that we manage).

I see, thanks for clearing it up.

But, what about giving students a branch in the svn and instructing
them to commit only there ? (Chris mentioned this two emails ago). If
for some reason svn is not the way to go, then I'm happy in using
bazaar for commits anyway.
Also, are you (PSF) planning to do this now or just at the "official"
gsoc start ? And, is there some internal discussion going on to decide
if students are going to get a branch or something at python
repositories ?

>
>  HTH,
>  Martin
>



-- 
-- Guilherme H. Polo Goncalves

From martin at v.loewis.de  Wed Apr 23 23:48:33 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 23 Apr 2008 23:48:33 +0200
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <ac2200130804231407l3671018bm4d21ccd364a60482@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>	<480ED121.7090005@v.loewis.de>	<ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>	<480F9DA8.4030103@v.loewis.de>
	<ac2200130804231407l3671018bm4d21ccd364a60482@mail.gmail.com>
Message-ID: <480FAEB1.2040500@v.loewis.de>

> But, what about giving students a branch in the svn and instructing
> them to commit only there ? (Chris mentioned this two emails ago). If
> for some reason svn is not the way to go, then I'm happy in using
> bazaar for commits anyway.

I think it's for the mentor to decide. I see no problems with svn -
it's just that Brett suggested to use bazaar.

> Also, are you (PSF) planning to do this now or just at the "official"
> gsoc start ? And, is there some internal discussion going on to decide
> if students are going to get a branch or something at python
> repositories ?

No, no discussion needed. The students should just send me their SSH
keys, and their preferred form of firstname.lastname (some people
may have names more complex than that), and I'll add them.

Regards,
Martin

From lists at cheimes.de  Wed Apr 23 23:51:53 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 23 Apr 2008 23:51:53 +0200
Subject: [Python-Dev] GSoC Student Introduction
In-Reply-To: <ac2200130804231407l3671018bm4d21ccd364a60482@mail.gmail.com>
References: <1aff528e0804221742p2e6e9b21jfa0f7606fddf611e@mail.gmail.com>	<1afaf6160804221938j1a6672cdv938fc373e00068f@mail.gmail.com>	<bbaeab100804222045r16a7923br8bfe0e64c0f5373b@mail.gmail.com>	<480ED121.7090005@v.loewis.de>	<ac2200130804230324h643b5401i83bbf0b8c2496fa4@mail.gmail.com>	<480F9DA8.4030103@v.loewis.de>
	<ac2200130804231407l3671018bm4d21ccd364a60482@mail.gmail.com>
Message-ID: <fuob1p$hoj$1@ger.gmane.org>

Guilherme Polo schrieb:
> Also, are you (PSF) planning to do this now or just at the "official"
> gsoc start ? And, is there some internal discussion going on to decide
> if students are going to get a branch or something at python
> repositories ?

I suggest we use the bounding period to set up the infrastructure for
the students. I've already written a private mail to my student with
several small tasks like subscribing to mailing lists, creating accounts
at bugs.python.org + wiki.python.org, mailing the contribution agreement
etc.

By the end of the bounding period everything should be in place -
including the working environment on the students' computers.

Christian


From bvidinli at gmail.com  Thu Apr 24 10:13:25 2008
From: bvidinli at gmail.com (bvidinli)
Date: Thu, 24 Apr 2008 11:13:25 +0300
Subject: [Python-Dev] annoying dictionary problem, non-existing keys
Message-ID: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>

i use dictionaries to hold some config data,
such as:

conf={'key1':'value1','key2':'value2'}
and so on...

when i try to process conf, i have to code every time like:
if conf.has_key('key1'):
     if conf['key1']<>'':
         other commands....


this is very annoying.
in php, i was able to code only like:
if conf['key1']=='someth'

in python, this fails, because, if key1 does not exists, it raises an exception.

MY question:
is there a way to directly get value of an array/tuple/dict  item by key,
as in php above, even if key may not exist,  i should not check if key exist,
i should only use it, if it does not exist, it may return only empty,
just as in php....

i hope you understand my question...

-- 
?.Bahattin Vidinli
Elk-Elektronik M?h.
-------------------
iletisim bilgileri (Tercih sirasina gore):
skype: bvidinli (sesli gorusme icin, www.skype.com)
msn: bvidinli at iyibirisi.com
yahoo: bvidinli

+90.532.7990607
+90.505.5667711

From tjreedy at udel.edu  Thu Apr 24 10:17:53 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 24 Apr 2008 04:17:53 -0400
Subject: [Python-Dev] annoying dictionary problem, non-existing keys
References: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>
Message-ID: <fupfne$heg$1@ger.gmane.org>

Python-dev is for discussion of development of future Python.  Use 
python-list / comp.lang.python / gmane.comp.python.general for usage 
questions. 




From martin at v.loewis.de  Thu Apr 24 10:20:38 2008
From: martin at v.loewis.de (=?ISO-8859-9?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 24 Apr 2008 10:20:38 +0200
Subject: [Python-Dev] annoying dictionary problem, non-existing keys
In-Reply-To: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>
References: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>
Message-ID: <481042D6.9000307@v.loewis.de>

> this is very annoying.

Posting to so many lists is even more annoying. You might not get
a single helpful answer just because people are so frustrated by
this behavior.

Regards,
Martin

From bvidinli at gmail.com  Thu Apr 24 10:36:52 2008
From: bvidinli at gmail.com (bvidinli)
Date: Thu, 24 Apr 2008 11:36:52 +0300
Subject: [Python-Dev] annoying dictionary problem, non-existing keys
In-Reply-To: <fupfne$heg$1@ger.gmane.org>
References: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>
	<fupfne$heg$1@ger.gmane.org>
Message-ID: <36e8a7020804240136r6e42f9fdv1f3d4dc3e947169a@mail.gmail.com>

I posted to so many lists because,

this issue is related to all lists,
this is an idea for python,
this is related to development of python...

why are you so much defensive ?

i think ideas all important for development of python, software....
i am sory anyway.... hope will be helpful.

2008/4/24, Terry Reedy <tjreedy at udel.edu>:
> Python-dev is for discussion of development of future Python.  Use
>  python-list / comp.lang.python / gmane.comp.python.general for usage
>  questions.
>
>
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/bvidinli%40gmail.com
>


-- 
?.Bahattin Vidinli
Elk-Elektronik M?h.
-------------------
iletisim bilgileri (Tercih sirasina gore):
skype: bvidinli (sesli gorusme icin, www.skype.com)
msn: bvidinli at iyibirisi.com
yahoo: bvidinli

+90.532.7990607
+90.505.5667711

From martin at v.loewis.de  Thu Apr 24 13:09:46 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 24 Apr 2008 13:09:46 +0200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>	<20080227141322.GA5871@amk-desktop.matrixgroup.net>	<47CDDF57.1020006@argo.es>
	<47CF0B7C.109@v.loewis.de>	<480CABFB.2080302@argo.es>	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>	<480EA87D.5040107@argo.es>
	<52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com>
Message-ID: <48106A7A.2090405@v.loewis.de>

> After that, merging his changes into trunk will be relatively easy and I
> think we should give jcea commit access and let him do it and henceforth
> use python svn as the official repository for the pybsddb code and
> documentation.

Ok. Jesus, can you please send me your SSH key?

> I do not see this as a daunting task assuming jcea is willing to be
> volunteered to do the merge and to handle any py3k merge issues that
> come up.

Ideally, yes. Jesus, if you are uncertain of how exactly the merging
works, don't hesitate to ask.

Regards,
Martin

From tnelson at onresolve.com  Thu Apr 24 15:34:53 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 24 Apr 2008 06:34:53 -0700
Subject: [Python-Dev] BSDDB3
In-Reply-To: <480EA87D.5040107@argo.es>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>
	<fpuf2u$6se$1@ger.gmane.org>
	<e04bdf310802251009u4d4af742t37da916fc0445e50@mail.gmail.com>
	<e04bdf310802251203w9b48a75qcaa1b03508e654dd@mail.gmail.com>
	<47C32D2D.1030206@free.fr>
	<e04bdf310802260322i5b44b12r65efef8cd74a846e@mail.gmail.com>
	<47C4813F.2080403@v.loewis.de>
	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>
	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>
	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>
	<20080227141322.GA5871@amk-desktop.matrixgroup.net>
	<47CDDF57.1020006@argo.es>	<47CF0B7C.109@v.loewis.de>
	<480CABFB.2080302@argo.es>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
	<480EA87D.5040107@argo.es>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2456D05A@EXMBX04.exchhosting.com>


> Next week I will (if nothing goes wrong) publish pybsddb
> 4.6.4. This release supports distributed transactions and
> replication, testsuite is way faster, and rewritten to be
> able to launch tests from multiple threads/processes if you
> wish, setuptools/pypi support, etc.

Great!  Once you've settled in with your changes let me know and I'll help with doing the necessary things on the Windows-side to set up the bsddb46.vcproj and switching the build to use that.

        Trent.

From greg.ewing at canterbury.ac.nz  Fri Apr 25 01:34:27 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Apr 2008 11:34:27 +1200
Subject: [Python-Dev] BSDDB3
In-Reply-To: <48106A7A.2090405@v.loewis.de>
References: <e04bdf310802250423o7e8b5ccbu6d0b48af468583e1@mail.gmail.com>
	<e04bdf310802261352n6a32183el389095f92e551355@mail.gmail.com>
	<bbaeab100802261404w7c670070tfb3997cd2351c097@mail.gmail.com>
	<52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com>
	<20080227141322.GA5871@amk-desktop.matrixgroup.net>
	<47CDDF57.1020006@argo.es>
	<47CF0B7C.109@v.loewis.de> <480CABFB.2080302@argo.es>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com>
	<480EA87D.5040107@argo.es>
	<52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com>
	<48106A7A.2090405@v.loewis.de>
Message-ID: <48111903.8020906@canterbury.ac.nz>

Martin v. L?wis wrote:

> Ok. Jesus, can you please send me your SSH key?

Nice to see that Heaven is security conscious and
accepting encrypted prayer nowadays. :-)

-- 
Greg

From nnorwitz at gmail.com  Fri Apr 25 06:49:24 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 24 Apr 2008 21:49:24 -0700
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <4807D51A.2060104@voidspace.org.uk>
References: <4807555B.2070206@voidspace.org.uk>
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>
	<480765B5.2010904@voidspace.org.uk> <fu8201$b07$1@ger.gmane.org>
	<4807D51A.2060104@voidspace.org.uk>
Message-ID: <ee2a432c0804242149y63799da5j8507b624febea665@mail.gmail.com>

[Michael working on cleaning up the unittest module]

it seems like most of the good ideas have been captured already.  I'll
through two more (low priority) ideas out there.

 1) Randomized test runner/option that runs tests in a random order
(like regrtest.py -r, but for methods)
 2) decorator to verify a test method is supposed to fail

#2 is useful for getting test cases into the code sooner rather than
later.  I'm pretty sure I have a patch that implements this
(http://bugs.python.org/issue1399935).  It didn't fit in well with the
old unittest structure, but seems closer to the direction you are
headed.

One other idea that probably ought not be done just yet:  add a way of
failing with the test continuing.  We use this at work (not in Python
though) and when used appropriately, it works quite well.  It provides
more information about the failure.  It looks something like this:

  def testMethod(self):
    # setup
    self.assertTrue(precondition)
    self.expectTrue(value)
    self.expectEqual(expected_result, other_value)

All the expect methods duplicate the assert methods.  Asserts can the
test to fail immediately, expects don't fail immediately allowing the
test to continue.  All the expect failures are collected and printed
at the end of the method run.  I was a little skeptical about assert
vs expect at first, but it has proven useful in the long run.  As I
said, I don't think this should be done now, maybe later.

n

From ncoghlan at gmail.com  Fri Apr 25 08:54:34 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 25 Apr 2008 16:54:34 +1000
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ee2a432c0804242149y63799da5j8507b624febea665@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>	<480765B5.2010904@voidspace.org.uk>
	<fu8201$b07$1@ger.gmane.org>	<4807D51A.2060104@voidspace.org.uk>
	<ee2a432c0804242149y63799da5j8507b624febea665@mail.gmail.com>
Message-ID: <4811802A.3030802@gmail.com>

Neal Norwitz wrote:
> All the expect methods duplicate the assert methods.  Asserts can the
> test to fail immediately, expects don't fail immediately allowing the
> test to continue.  All the expect failures are collected and printed
> at the end of the method run.  I was a little skeptical about assert
> vs expect at first, but it has proven useful in the long run.  As I
> said, I don't think this should be done now, maybe later.

I'd agree these kinds of things can be very useful, particularly for 
tests which run the same test for multiple inputs - when those tests die 
on the first set of input values that do the wrong thing, it greatly 
slows down the test-fix-retest cycle, since you have to go through it 
multiple times. If the test is written to test all the input values, 
even if some of them fail, and then report at the end "test failed, here 
are the (input, expected_ouput, actual_output) that didn't work" it can 
make tests far more efficient.

At the moment I do it manually by collected a list of failed triples, 
then reporting a test failure at the end if the list isn't empty, but 
something like "expect" methods would avoid the need to recreate that 
infrastructure every time I need it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fuzzyman at voidspace.org.uk  Fri Apr 25 11:36:14 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 25 Apr 2008 10:36:14 +0100
Subject: [Python-Dev] Proposed unittest changes
In-Reply-To: <ee2a432c0804242149y63799da5j8507b624febea665@mail.gmail.com>
References: <4807555B.2070206@voidspace.org.uk>	
	<ca471dc20804170754y7e689791m41f99698a41e406f@mail.gmail.com>	
	<480765B5.2010904@voidspace.org.uk> <fu8201$b07$1@ger.gmane.org>	
	<4807D51A.2060104@voidspace.org.uk>
	<ee2a432c0804242149y63799da5j8507b624febea665@mail.gmail.com>
Message-ID: <4811A60E.6010207@voidspace.org.uk>

Neal Norwitz wrote:
> [Michael working on cleaning up the unittest module]
>
> it seems like most of the good ideas have been captured already.  I'll
> through two more (low priority) ideas out there.
>
>  1) Randomized test runner/option that runs tests in a random order
> (like regrtest.py -r, but for methods)
>   

Should be easy enough.

>  2) decorator to verify a test method is supposed to fail
>
> #2 is useful for getting test cases into the code sooner rather than
> later.  I'm pretty sure I have a patch that implements this
> (http://bugs.python.org/issue1399935).  It didn't fit in well with the
> old unittest structure, but seems closer to the direction you are
> headed.
>   


We normally just prefix the test name with 'DONT' so that it isn't 
actually run...

> One other idea that probably ought not be done just yet:  add a way of
> failing with the test continuing.  We use this at work (not in Python
> though) and when used appropriately, it works quite well.  It provides
> more information about the failure.  It looks something like this:
>
>   def testMethod(self):
>     # setup
>     self.assertTrue(precondition)
>     self.expectTrue(value)
>     self.expectEqual(expected_result, other_value)
>
> All the expect methods duplicate the assert methods.  Asserts can the
> test to fail immediately, expects don't fail immediately allowing the
> test to continue.  All the expect failures are collected and printed
> at the end of the method run.  I was a little skeptical about assert
> vs expect at first, but it has proven useful in the long run.  As I
> said, I don't think this should be done now, maybe later.
>
> n
>   
I like this pattern.

Michael Foord

From ndbecker2 at gmail.com  Fri Apr 25 15:13:54 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 25 Apr 2008 09:13:54 -0400
Subject: [Python-Dev] segfault in 2.5.1
Message-ID: <fuslei$a41$1@ger.gmane.org>

Attempt to write to a mmap which was opened mmap.PROT_READ causes python to
segfault.


From schmir at gmail.com  Fri Apr 25 15:26:50 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Fri, 25 Apr 2008 15:26:50 +0200
Subject: [Python-Dev] segfault in 2.5.1
In-Reply-To: <fuslei$a41$1@ger.gmane.org>
References: <fuslei$a41$1@ger.gmane.org>
Message-ID: <932f8baf0804250626i5924a1dcvb8ffadbb74a4850d@mail.gmail.com>

On Fri, Apr 25, 2008 at 3:13 PM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Attempt to write to a mmap which was opened mmap.PROT_READ causes python
> to
> segfault.


 http://bugs.python.org/issue2111
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080425/7ab82458/attachment.htm>

From ndbecker2 at gmail.com  Fri Apr 25 15:32:33 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 25 Apr 2008 09:32:33 -0400
Subject: [Python-Dev] mmap documentation error (2.5.1)
Message-ID: <fusmhj$eiu$1@ger.gmane.org>

mmap(  fileno, length[, flags[, prot[, access]]])
(Unix version) Maps length bytes from the file specified by the file
descriptor fileno, and returns a mmap object. If length is 0, the maximum
length of the map will be the current size of the file when mmap() is
called. 
flags specifies the nature of the mapping. MAP_PRIVATE creates a private
copy-on-write mapping, so changes to the contents of the mmap object will
be private to this process, and MAP_SHARED creates a mapping that's shared
with all other processes mapping the same areas of the file. The default
value is MAP_SHARED. 

Apparantly, this is wrong - the default is not MAP_SHARED, as I just found
out the hard way.


From ndbecker2 at gmail.com  Fri Apr 25 15:41:12 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 25 Apr 2008 09:41:12 -0400
Subject: [Python-Dev] mmap documentation error (2.5.1)
References: <fusmhj$eiu$1@ger.gmane.org>
Message-ID: <fusn1o$ga5$1@ger.gmane.org>

Sorry, my mistake.  Acutally, I was trying to debug this:
On linux, I don't understand why:

f = open ('/dev/eos', 'rw')
m = mmap.mmap(f.fileno(), 1000000, prot=mmap.PROT_READ|mmap.PROT_WRITE,
flags=mmap.MAP_SHARED)

gives 'permission denied', but this c++ code works:

#include <sys/mman.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <iostream>

int main() {
  int fd = open ("/dev/eos", O_RDWR);
  void* m = mmap (NULL, 1000000, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
  std::cout << m << '\n';
}


From aahz at pythoncraft.com  Fri Apr 25 16:38:22 2008
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 25 Apr 2008 07:38:22 -0700
Subject: [Python-Dev] mmap documentation error (2.5.1)
In-Reply-To: <fusn1o$ga5$1@ger.gmane.org>
References: <fusmhj$eiu$1@ger.gmane.org> <fusn1o$ga5$1@ger.gmane.org>
Message-ID: <20080425143822.GA25649@panix.com>

On Fri, Apr 25, 2008, Neal Becker wrote:
>
> Sorry, my mistake.  Acutally, I was trying to debug this:

python-dev is probably not the right place for this -- please use c.l.py
or the list capi-sig.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Why is this newsgroup different from all other newsgroups?  

From status at bugs.python.org  Fri Apr 25 18:06:36 2008
From: status at bugs.python.org (Tracker)
Date: Fri, 25 Apr 2008 18:06:36 +0200 (CEST)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20080425160636.37FEE78300@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (04/18/08 - 04/25/08)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1850 open (+26) / 12667 closed (+11) / 14517 total (+37)

Open issues with patches:   558

Average duration of open issues: 707 days.
Median duration of open issues: 1333 days.

Open Issues Breakdown
   open  1828 (+26)
pending    22 ( +0)

Issues Created Or Reopened (37)
_______________________________

Create ctypes instances from buffer interface                    04/18/08
CLOSED http://bugs.python.org/issue2655    created  theller                   
       patch, patch                                                            

Autodoc should skip inherited methods                            04/18/08
       http://bugs.python.org/issue2656    created  jmillikin                 
                                                                               

curses                                                           04/18/08
       http://bugs.python.org/issue2657    created  fer_perez                 
                                                                               

decode_header() fails on multiline headers                       04/19/08
       http://bugs.python.org/issue2658    created  cschnee                   
                                                                               

textwrap handling of hyphenation                                 04/19/08
       http://bugs.python.org/issue2659    created  fourmanoit                
       patch                                                                   

Py3k fails to parse a file with an iso-8859-1 string             04/19/08
       http://bugs.python.org/issue2660    created  azverkan                  
                                                                               

Mapping tests cannot be passed by user implementations           04/19/08
       http://bugs.python.org/issue2661    created  danderson                 
                                                                               

Allows HUGE_VAL as extern variable                               04/20/08
CLOSED http://bugs.python.org/issue2662    created  ocean-city                
       patch                                                                   

shutil.copytree glob-style filtering [patch]                     04/20/08
       http://bugs.python.org/issue2663    created  tarek                     
       patch                                                                   

The position of special value tables (cmathmodule.c)             04/21/08
       http://bugs.python.org/issue2664    created  ocean-city                
       patch                                                                   

unable to launch IDLE on windows XP                              04/21/08
CLOSED http://bugs.python.org/issue2665    created  Leo                       
                                                                               

webbrowser.py doesn't properly handle BROWSER env var            04/21/08
       http://bugs.python.org/issue2666    created  phd                       
       patch                                                                   

Remove multiple inheritance in Python 3000                       04/22/08
CLOSED http://bugs.python.org/issue2667    created  gmarketer                 
                                                                               

apply() documentation is slightly incorrect                      04/22/08
       http://bugs.python.org/issue2668    created  belopolsky                
                                                                               

bsddb iterkeys sliently fails when database modified during iter 04/22/08
       http://bugs.python.org/issue2669    created  tebeka                    
                                                                               

urllib2 build_opener() fails if two handlers use the same defaul 04/22/08
CLOSED http://bugs.python.org/issue2670    created  tlynn                     
                                                                               

PyErr_WarnPy3k                                                   04/22/08
       http://bugs.python.org/issue2671    created  benjamin.peterson         
       patch                                                                   

speed of set.update(                                             04/23/08
CLOSED http://bugs.python.org/issue2672    created  jameinel                  
                                                                               

error on optparse page                                           04/23/08
CLOSED http://bugs.python.org/issue2673    created  fbe2                      
                                                                               

unittest.TestProgram uses sys.exit()                             04/23/08
       http://bugs.python.org/issue2674    created  tuben                     
                                                                               

Curses terminal resize problems when Python is in interactive mo 04/23/08
       http://bugs.python.org/issue2675    created  pbazant                   
                                                                               

email/message.py [Message.get_content_type]: Trivial regex hangs 04/23/08
       http://bugs.python.org/issue2676    created  ajaksu2                   
       patch                                                                   

Argument rules for callables do not apply when function implemen 04/24/08
       http://bugs.python.org/issue2677    created  garden                    
                                                                               

hmac performance optimization                                    04/24/08
CLOSED http://bugs.python.org/issue2678    created  fafhrd                    
       patch                                                                   

email.feedparser regex duplicate                                 04/24/08
       http://bugs.python.org/issue2679    created  jimjjewett                
                                                                               

gotcha: _fields_ is final but accepts lists                      04/24/08
CLOSED http://bugs.python.org/issue2680    created  cscheid                   
                                                                               

octal literals beginning with 8 don't raise a SyntaxError        04/24/08
CLOSED http://bugs.python.org/issue2681    created  lukas.meuser              
       patch                                                                   

cyclic reference in ctypes CFunctionType objects                 04/24/08
       http://bugs.python.org/issue2682    created  theller                   
       patch                                                                   

subprocess.Popen.communicate takes bytes, not str                04/24/08
       http://bugs.python.org/issue2683    created  kermode                   
                                                                               

Logging Module still failing for %(filename)s, __init__          04/24/08
       http://bugs.python.org/issue2684    created  CharlesMerriam            
                                                                               

Add -mieee to compile flags, when available                      04/24/08
       http://bugs.python.org/issue2685    created  marketdickinson           
       patch                                                                   

Any chance we could double the height of the 'Comment:' text are 04/24/08
CLOSED http://bugs.python.org/issue2686    created  Trent.Nelson              
                                                                               

SSL example script fails mysteriously on MacOS                   04/25/08
       http://bugs.python.org/issue2687    created  akuchling                 
                                                                               

Error when nesting many while loops                              04/25/08
       http://bugs.python.org/issue2688    created  rgreimel                  
                                                                               

Fix indentation in range_item                                    04/25/08
       http://bugs.python.org/issue2689    created  belopolsky                
       patch                                                                   

Precompute range length                                          04/25/08
       http://bugs.python.org/issue2690    created  belopolsky                
       patch                                                                   

Document size_t related long object APIs                         04/25/08
       http://bugs.python.org/issue2691    created  belopolsky                
                                                                               



Issues Now Closed (27)
______________________

Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmcl  180 days
       http://bugs.python.org/issue1327    amaury.forgeotdarc        
                                                                               

cmath is numerically unsound                                      172 days
       http://bugs.python.org/issue1381    marketdickinson           
       patch                                                                   

sqrt(-1) doesn't raise ValueError on OS X                          92 days
       http://bugs.python.org/issue1879    marketdickinson           
       patch                                                                   

xmlrpclib cannot send datetime objects with dates before 1900      75 days
       http://bugs.python.org/issue2014    schmir                    
       patch                                                                   

Restructure import.c into PEP 302 importer objects                 64 days
       http://bugs.python.org/issue2135    brett.cannon              
       patch                                                                   

% string formatting does not support %b                            31 days
       http://bugs.python.org/issue2416    eric.smith                
                                                                               

Issues with getargs_n() and PyNumber_Index.                        33 days
       http://bugs.python.org/issue2440    Trent.Nelson              
       patch, patch, 64bit                                                     

imp.get_magic() should return bytes, not bytearray                 32 days
       http://bugs.python.org/issue2471    amaury.forgeotdarc        
       patch                                                                   

IMPORT_NAME Documentation is incomplete                             5 days
       http://bugs.python.org/issue2631    georg.brandl              
                                                                               

Improve subprocess.Popen() documentation ("env" parameter)          5 days
       http://bugs.python.org/issue2633    georg.brandl              
                                                                               

os.execvpe() docs need to be more specific                          5 days
       http://bugs.python.org/issue2634    georg.brandl              
                                                                               

shutil.copyfile() documentation is vague                            4 days
       http://bugs.python.org/issue2639    georg.brandl              
                                                                               

httplib throws ValueError                                           4 days
       http://bugs.python.org/issue2645    georg.brandl              
       patch                                                                   

Typo at http://docs.python.org/dev/3.0/howto/doanddont.html         0 days
       http://bugs.python.org/issue2654    akuchling                 
                                                                               

Create ctypes instances from buffer interface                       7 days
       http://bugs.python.org/issue2655    theller                   
       patch, patch                                                            

Allows HUGE_VAL as extern variable                                  0 days
       http://bugs.python.org/issue2662    marketdickinson           
       patch                                                                   

unable to launch IDLE on windows XP                                 1 days
       http://bugs.python.org/issue2665    amaury.forgeotdarc        
                                                                               

Remove multiple inheritance in Python 3000                          0 days
       http://bugs.python.org/issue2667    gmarketer                 
                                                                               

urllib2 build_opener() fails if two handlers use the same defaul    0 days
       http://bugs.python.org/issue2670    amaury.forgeotdarc        
                                                                               

speed of set.update(                                                2 days
       http://bugs.python.org/issue2672    jameinel                  
                                                                               

error on optparse page                                              1 days
       http://bugs.python.org/issue2673    benjamin.peterson         
                                                                               

hmac performance optimization                                       0 days
       http://bugs.python.org/issue2678    amaury.forgeotdarc        
       patch                                                                   

gotcha: _fields_ is final but accepts lists                         0 days
       http://bugs.python.org/issue2680    theller                   
                                                                               

octal literals beginning with 8 don't raise a SyntaxError           0 days
       http://bugs.python.org/issue2681    amaury.forgeotdarc        
       patch                                                                   

Any chance we could double the height of the 'Comment:' text are    0 days
       http://bugs.python.org/issue2686    Trent.Nelson              
                                                                               

tokenize module w/ coding cookie                                 1838 days
       http://bugs.python.org/issue719888  Trent.Nelson              
       patch                                                                   

xmlrpclib.Binary doesn't say which base64 spec it implements      285 days
       http://bugs.python.org/issue1753732 benjamin.peterson         
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 25 Write user's version of the reference guide                      869 days
open    http://bugs.python.org/issue1376292

  7 Argument rules for callables do not apply when function impleme    1 days
open    http://bugs.python.org/issue2677   

  6 speed of set.update(                                               2 days
closed  http://bugs.python.org/issue2672   

  5 unable to launch IDLE on windows XP                                1 days
closed  http://bugs.python.org/issue2665   

  4 cyclic reference in ctypes CFunctionType objects                   1 days
open    http://bugs.python.org/issue2682   

  4 httplib throws ValueError                                          4 days
closed  http://bugs.python.org/issue2645   

  4 Regexp 2.6 (modifications to current re 2.2.2)                    10 days
open    http://bugs.python.org/issue2636   

  4 Adapt pydoc to new doc system                                     95 days
open    http://bugs.python.org/issue1883   

  3 Any chance we could double the height of the 'Comment:' text ar    0 days
closed  http://bugs.python.org/issue2686   

  3 gotcha: _fields_ is final but accepts lists                        0 days
closed  http://bugs.python.org/issue2680   




From tnelson at onresolve.com  Fri Apr 25 21:49:13 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 25 Apr 2008 12:49:13 -0700
Subject: [Python-Dev] Do we still need BaseAddress in .vcproj files?
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>


I just did this locally to all my .vcproj files for .pyds:

-  BaseAddress="0x1e000000"
+  RandomizedBaseAddress="2"

This was partly out of curiosity, and partly because it was quicker doing that than finding a new unique base address to use for a new module I added to my local tree.  Rebuilt everything and ran a full regression test, and everything passed.  What am I missing?  Do we have parts of Python that rely on finding modules at explicit base addresses?  Is BaseAddress a relic from the past?  Christian or Martin?

        Trent.

From lists at cheimes.de  Fri Apr 25 22:16:11 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 25 Apr 2008 22:16:11 +0200
Subject: [Python-Dev] Do we still need BaseAddress in .vcproj files?
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>
Message-ID: <48123C0B.10401@cheimes.de>

Trent Nelson schrieb:
> I just did this locally to all my .vcproj files for .pyds:
> 
> -  BaseAddress="0x1e000000"
> +  RandomizedBaseAddress="2"
> 
> This was partly out of curiosity, and partly because it was quicker doing that than finding a new unique base address to use for a new module I added to my local tree.  Rebuilt everything and ran a full regression test, and everything passed.  What am I missing?  Do we have parts of Python that rely on finding modules at explicit base addresses?  Is BaseAddress a relic from the past?  Christian or Martin?


An explicit base address speeds up the loading of shared extension
modules a bit. Checkout PC/dllbase_nt.txt

Christian

From martin at v.loewis.de  Fri Apr 25 23:00:18 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 25 Apr 2008 23:00:18 +0200
Subject: [Python-Dev] Do we still need BaseAddress in .vcproj files?
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>
Message-ID: <48124662.2080502@v.loewis.de>

> This was partly out of curiosity, and partly because it was quicker
> doing that than finding a new unique base address to use for a new
> module I added to my local tree.  Rebuilt everything and ran a full
> regression test, and everything passed.  What am I missing?  Do we
> have parts of Python that rely on finding modules at explicit base
> addresses?  Is BaseAddress a relic from the past?  Christian or
> Martin?

As Christian says: specifying the base address so that the addresses
don't overlap avoids having Windows to perform relocations on startup.

There is a Microsoft tool (editbin /rebase) to compute non-overlapping
base addresses for a given set of DLLs.

Regards,
Martin


From varmaa at gmail.com  Fri Apr 25 23:52:31 2008
From: varmaa at gmail.com (Atul Varma)
Date: Fri, 25 Apr 2008 14:52:31 -0700
Subject: [Python-Dev] Do we still need BaseAddress in .vcproj files?
In-Reply-To: <48124662.2080502@v.loewis.de>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E246919B5@EXMBX04.exchhosting.com>
	<48124662.2080502@v.loewis.de>
Message-ID: <361b27370804251452r3a17cb58p8fedf5c9461e65c8@mail.gmail.com>

A while back I was reading an MSDN article that did some concrete research
on the performance benefits of rebasing DLLs to avoid fixups at runtime, and
it actually concluded that on modern systems, the benefits are really
negligible.  I tried finding the article on the net just now, though, and I
couldn't, so don't take my word for it.  This 1995 MSDN article provides an
explanation and some raw numbers for systems at that time, though:

  http://msdn2.microsoft.com/en-us/library/ms810432.aspx

- Atul

On Fri, Apr 25, 2008 at 2:00 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > This was partly out of curiosity, and partly because it was quicker
> > doing that than finding a new unique base address to use for a new
> > module I added to my local tree.  Rebuilt everything and ran a full
> > regression test, and everything passed.  What am I missing?  Do we
> > have parts of Python that rely on finding modules at explicit base
> > addresses?  Is BaseAddress a relic from the past?  Christian or
> > Martin?
>
> As Christian says: specifying the base address so that the addresses
> don't overlap avoids having Windows to perform relocations on startup.
>
> There is a Microsoft tool (editbin /rebase) to compute non-overlapping
> base addresses for a given set of DLLs.
>
> Regards,
> Martin
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/varmaa%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080425/32942e94/attachment.htm>

From collinw at gmail.com  Sat Apr 26 01:52:23 2008
From: collinw at gmail.com (Collin Winter)
Date: Fri, 25 Apr 2008 16:52:23 -0700
Subject: [Python-Dev] annoying dictionary problem, non-existing keys
In-Reply-To: <36e8a7020804240136r6e42f9fdv1f3d4dc3e947169a@mail.gmail.com>
References: <36e8a7020804240113m7cbd10efibfe9f9b68f6135ac@mail.gmail.com>
	<fupfne$heg$1@ger.gmane.org>
	<36e8a7020804240136r6e42f9fdv1f3d4dc3e947169a@mail.gmail.com>
Message-ID: <43aa6ff70804251652r4f0deb39he5b2e1c71e8aabee@mail.gmail.com>

2008/4/24 bvidinli <bvidinli at gmail.com>:
> I posted to so many lists because,
>
>  this issue is related to all lists,
>  this is an idea for python,
>  this is related to development of python...
>
>  why are you so much defensive ?
>
>  i think ideas all important for development of python, software....
>  i am sory anyway.... hope will be helpful.

Please consult the documentation first:
http://docs.python.org/lib/typesmapping.html . You're looking for the
get() method.

This attribute of PHP is hardly considered a feature, and is not
something Python wishes to emulate.

Collin Winter

>  2008/4/24, Terry Reedy <tjreedy at udel.edu>:
>
>
> > Python-dev is for discussion of development of future Python.  Use
>  >  python-list / comp.lang.python / gmane.comp.python.general for usage
>  >  questions.
>  >
>  >
>  >
>  >  _______________________________________________
>  >  Python-Dev mailing list
>  >  Python-Dev at python.org
>  >  http://mail.python.org/mailman/listinfo/python-dev
>  >  Unsubscribe: http://mail.python.org/mailman/options/python-dev/bvidinli%40gmail.com
>
> >
>
>
>  --
>  ?.Bahattin Vidinli
>  Elk-Elektronik M?h.
>  -------------------
>  iletisim bilgileri (Tercih sirasina gore):
>  skype: bvidinli (sesli gorusme icin, www.skype.com)
>  msn: bvidinli at iyibirisi.com
>  yahoo: bvidinli
>
>  +90.532.7990607
>  +90.505.5667711
>  _______________________________________________
>
>
> Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/collinw%40gmail.com
>

From musiccomposition at gmail.com  Sun Apr 27 02:33:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sat, 26 Apr 2008 19:33:08 -0500
Subject: [Python-Dev] PyErr_Py3kWarn
Message-ID: <1afaf6160804261733t41d8fde8pef16a749828ac766@mail.gmail.com>

I have a patch to to add an easy way to give a Py3k warning (issue
2671). Does this sound useful?

-- 
Cheers,
Benjamin Peterson

From brett at python.org  Sun Apr 27 22:40:52 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Apr 2008 13:40:52 -0700
Subject: [Python-Dev] Dealing with a desired change to warnings.showwarning()
Message-ID: <bbaeab100804271340q21b4a364n41c8e4c62c8d3706@mail.gmail.com>

As part of my rewrite of warnings into C, I added a new, optional
argument to showwarning(): line, which defaults to None.

As http://bugs.python.org/issue2705 points out, though, since the
function has been documented as being allowed to be overridden, this
potentially breaks existing showwarning() implementations. That means
something needs to change.

One option is to introduce a new function. But what to name it?
show_warning()? That small of a name change seems like asking for
trouble, but showwarning_ex() just seems wrong to use in Python code.

Another is that where showwarning() is called within the code, if a
TypeError is raised or introspection proves the call can't work, then
to try again without the 'line' argument, and raise a deprecation
warning.

Or try to re-architect _warnings.c to not use the 'line' argument
(although it is handy for testing purposes).

Anyway, people have any suggestions?

-Brett

From skip at pobox.com  Mon Apr 28 00:16:12 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 27 Apr 2008 17:16:12 -0500
Subject: [Python-Dev] Double specification of function signatures?
Message-ID: <18452.64300.579813.77697@montanaro-dyndns-org.local>

While cleaning up the documentation for the tempfile module I noticed that
the docstrings for the mk*temp functions in the module itself list their
signatures (incompletely) in the first line.  I don't know if that was
intentional, but it seems both redundant and error-prone to me.  The help()
function already displays the signatures of Python functions.  There's no
need to put them in docstrings and risk having them out-of-date.  For
example:

    >>> help(tempfile.mkdtemp)
    Help on function mkdtemp in module tempfile:

    mkdtemp(suffix='', prefix='tmp', dir=None)
        mkdtemp([suffix, [prefix, [dir]]])
        User-callable function to create and return a unique temporary
        directory.  The return value is the pathname of the directory.

Am I way off-base here?  Let me know, as I have a couple minor tweaks to
check in besides these.

Thx,

Skip

From fuzzyman at voidspace.org.uk  Mon Apr 28 00:26:29 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 27 Apr 2008 23:26:29 +0100
Subject: [Python-Dev] [Doc-SIG] Double specification of function
	signatures?
In-Reply-To: <18452.64300.579813.77697@montanaro-dyndns-org.local>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
Message-ID: <4814FD95.2010206@voidspace.org.uk>

skip at pobox.com wrote:
> While cleaning up the documentation for the tempfile module I noticed that
> the docstrings for the mk*temp functions in the module itself list their
> signatures (incompletely) in the first line.  I don't know if that was
> intentional, but it seems both redundant and error-prone to me.  The help()
> function already displays the signatures of Python functions.  There's no
> need to put them in docstrings and risk having them out-of-date.  For
> example:
>
>     >>> help(tempfile.mkdtemp)
>     Help on function mkdtemp in module tempfile:
>
>     mkdtemp(suffix='', prefix='tmp', dir=None)
>         mkdtemp([suffix, [prefix, [dir]]])
>         User-callable function to create and return a unique temporary
>         directory.  The return value is the pathname of the directory.
>
> Am I way off-base here?  Let me know, as I have a couple minor tweaks to
> check in besides these.
>   

It seems that any documentation or help tool worth its salt should fetch 
the parameters from the definition and so including them in the 
docstring should be redundant duplication.

Michael Foord

> Thx,
>
> Skip
> _______________________________________________
> Doc-SIG maillist  -  Doc-SIG at python.org
> http://mail.python.org/mailman/listinfo/doc-sig
>   


From skip at pobox.com  Mon Apr 28 00:29:33 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 27 Apr 2008 17:29:33 -0500
Subject: [Python-Dev] [Doc-SIG] Double specification of function
	signatures?
In-Reply-To: <4814FD95.2010206@voidspace.org.uk>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
Message-ID: <18452.65101.202393.473799@montanaro-dyndns-org.local>


    Michael> It seems that any documentation or help tool worth its salt
    Michael> should fetch the parameters from the definition and so
    Michael> including them in the docstring should be redundant
    Michael> duplication.

That's my position as well.  Currently we have no way to extract the
function signatures from C code on-the-fly or in a preprocessing step (might
be a good GSoC project), but for functions written in Python I'm of the
opinion the docstrings should not include function signatures.

Skip

From brett at python.org  Mon Apr 28 00:31:51 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Apr 2008 15:31:51 -0700
Subject: [Python-Dev] [Doc-SIG] Double specification of function
	signatures?
In-Reply-To: <18452.65101.202393.473799@montanaro-dyndns-org.local>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
	<18452.65101.202393.473799@montanaro-dyndns-org.local>
Message-ID: <bbaeab100804271531x1aeae74eic32a930bc43dcbfd@mail.gmail.com>

On Sun, Apr 27, 2008 at 3:29 PM,  <skip at pobox.com> wrote:
>
>     Michael> It seems that any documentation or help tool worth its salt
>     Michael> should fetch the parameters from the definition and so
>     Michael> including them in the docstring should be redundant
>     Michael> duplication.
>
>  That's my position as well.  Currently we have no way to extract the
>  function signatures from C code on-the-fly or in a preprocessing step (might
>  be a good GSoC project), but for functions written in Python I'm of the
>  opinion the docstrings should not include function signatures.

They shouldn't. Maybe the tempfile module came from a third-party that
had some internal doc rule of mentioning the call signature.
Regardless, just rip it out.

-Brett

From skip at pobox.com  Mon Apr 28 00:33:14 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 27 Apr 2008 17:33:14 -0500
Subject: [Python-Dev] Warn about mktemp once again?
Message-ID: <18452.65322.843226.299769@montanaro-dyndns-org.local>

Back in r29829, Guido commented out the security hole warning for
tempfile.mktemp:

    r29829 | gvanrossum | 2002-11-22 09:56:29 -0600 (Fri, 22 Nov 2002) | 3 lines

    Comment out the warnings about mktemp().  These are too annoying, and
    often unavoidable.

Any thought about whether this warning should be restored?  We're 5+ years
later.  Hopefully many uses of mktemp have been removed.  If we're not going
to restore the warning perhaps the commented code should just be deleted.

Skip


From stephen at xemacs.org  Mon Apr 28 01:40:02 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 28 Apr 2008 08:40:02 +0900
Subject: [Python-Dev] [Doc-SIG] Double specification of
	function	signatures?
In-Reply-To: <18452.65101.202393.473799@montanaro-dyndns-org.local>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
	<18452.65101.202393.473799@montanaro-dyndns-org.local>
Message-ID: <8763u2yii5.fsf@uwakimon.sk.tsukuba.ac.jp>

skip at pobox.com writes:

 > Currently we have no way to extract the function signatures from C
 > code on-the-fly or in a preprocessing step (might be a good GSoC
 > project),

+1 on GSoC, except for the bad timing (maybe somebody will start
sponsoring a "Winter Coding Sprint" event?)


From brett at python.org  Mon Apr 28 02:31:37 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Apr 2008 17:31:37 -0700
Subject: [Python-Dev] [Doc-SIG] Double specification of function
	signatures?
In-Reply-To: <8763u2yii5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
	<18452.65101.202393.473799@montanaro-dyndns-org.local>
	<8763u2yii5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <bbaeab100804271731x7b5f1214ob2cc5db34158a659@mail.gmail.com>

On Sun, Apr 27, 2008 at 4:40 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> skip at pobox.com writes:
>
>   > Currently we have no way to extract the function signatures from C
>   > code on-the-fly or in a preprocessing step (might be a good GSoC
>   > project),
>
>  +1 on GSoC, except for the bad timing (maybe somebody will start
>  sponsoring a "Winter Coding Sprint" event?)
>

It can still go on the wiki, though, so we don't forget about it.

-Brett

From skip at pobox.com  Mon Apr 28 05:05:11 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 27 Apr 2008 22:05:11 -0500
Subject: [Python-Dev] [Doc-SIG] Double specification of function
 signatures?
In-Reply-To: <bbaeab100804271731x7b5f1214ob2cc5db34158a659@mail.gmail.com>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
	<18452.65101.202393.473799@montanaro-dyndns-org.local>
	<8763u2yii5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<bbaeab100804271731x7b5f1214ob2cc5db34158a659@mail.gmail.com>
Message-ID: <18453.16103.592698.156270@montanaro-dyndns-org.local>

    >> > Currently we have no way to extract the function signatures from C
    >> > code on-the-fly or in a preprocessing step (might be a good GSoC
    >> > project),

    Brett> It can still go on the wiki, though, so we don't forget about it.

Done.

Skip


From skip at pobox.com  Mon Apr 28 05:06:06 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 27 Apr 2008 22:06:06 -0500
Subject: [Python-Dev] [Doc-SIG] Double specification of function
 signatures?
In-Reply-To: <bbaeab100804271531x1aeae74eic32a930bc43dcbfd@mail.gmail.com>
References: <18452.64300.579813.77697@montanaro-dyndns-org.local>
	<4814FD95.2010206@voidspace.org.uk>
	<18452.65101.202393.473799@montanaro-dyndns-org.local>
	<bbaeab100804271531x1aeae74eic32a930bc43dcbfd@mail.gmail.com>
Message-ID: <18453.16158.25027.879421@montanaro-dyndns-org.local>


    Brett> They shouldn't. Maybe the tempfile module came from a third-party
    Brett> that had some internal doc rule of mentioning the call signature.
    Brett> Regardless, just rip it out.

Done.

Skip

From mhammond at skippinet.com.au  Mon Apr 28 06:41:18 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Mon, 28 Apr 2008 14:41:18 +1000
Subject: [Python-Dev] Is Py_WIN_WIDE_FILENAMES still alive?
In-Reply-To: <000901c8a384$e3965570$0300a8c0@whiterabc2znlh>
References: <000901c8a384$e3965570$0300a8c0@whiterabc2znlh>
Message-ID: <005d01c8a8ea$1c954f60$55bfee20$@com.au>

> Hello. I noticed when I removes following line in trunk/PC/pyconfig.h
> 
> #define Py_WIN_WIDE_FILENAMES
> 
> _fileio.c and posixmodule.c (and maybe more) cannot be compiled on
> Windows.
> 
> When Py_WIN_WIDE_FILENAMES is not defined, how should python behave?
> 
>   - call posix functions like open(2)
> 
>   - call ANSI Win32 APIs like MoveFileA
> 
> Or maybe this macro is not used anymore?

I believe the macro should be removed, as Python currently assumes Unicode
APIs are available in a number of places.  This consistent with the versions
of Windows Python currently supports.  It is possible that patches would be
accepted which enable Python to be built without this functionality - in
which case the next-best behavior would probably be to convert Unicode to
MBCS encoded strings and call the *A versions of the API.  Looking at the
history for posixmodule.c etc will offer some clues as to how this could
best be done.

Cheers,

Mark


From mhammond at skippinet.com.au  Mon Apr 28 08:22:26 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Mon, 28 Apr 2008 16:22:26 +1000
Subject: [Python-Dev] Python 2.6a2 execution times with various compilers
In-Reply-To: <52dc1c820804121521u4c8f69d8q1bf0d0bb9b00e03a@mail.gmail.com>
References: <20080411151024.GJ51167@nexus.in-nomine.org>	<20080412180936.GP51167@nexus.in-nomine.org>
	<52dc1c820804121521u4c8f69d8q1bf0d0bb9b00e03a@mail.gmail.com>
Message-ID: <006f01c8a8f8$3d651690$b82f43b0$@com.au>

> Profile-guided optimization did not help much, as might be expected, it
> pushed about the same kind of optimization as the mtune/march combination.

> With gcc 4.1.3 i'm finding that profile guided optimization when trained 
> on pybench or regrtest does make a measurable difference (2-5% overall 
> time with 10-20% on some pybench tests).? I haven't run benchmarks enough 
> times to be confident in my results yet, I'll report back with data once 
> I have it.? I'm testing both pybench and regrtest as profiling training
runs.

It seems that profile guided optimization also offers some benefits on
Windows (eg,
http://mail.python.org/pipermail/python-dev/2007-May/072970.html), so it
might be worth trying to coordinate such efforts between platforms (eg, a
central document which holds results for all supported platforms sounds
worthwhile, and maybe sharing the top-level script that generates the
profile data, etc) 

Cheers,

Mark


From ocean at m2.ccsnet.ne.jp  Mon Apr 28 08:32:59 2008
From: ocean at m2.ccsnet.ne.jp (ocean)
Date: Mon, 28 Apr 2008 15:32:59 +0900
Subject: [Python-Dev] Is Py_WIN_WIDE_FILENAMES still alive?
References: <000901c8a384$e3965570$0300a8c0@whiterabc2znlh>
	<005d01c8a8ea$1c954f60$55bfee20$@com.au>
Message-ID: <000701c8a8f9$b4487670$0200a8c0@whiterabc2znlh>

> I believe the macro should be removed, as Python currently assumes Unicode
> APIs are available in a number of places.

My +1 for removal. Even 2.5 cannot be compiled without this macro, probably
no one is using this.

> This consistent with the versions
> of Windows Python currently supports.  It is possible that patches would
be
> accepted which enable Python to be built without this functionality - in
> which case the next-best behavior would probably be to convert Unicode to
> MBCS encoded strings and call the *A versions of the API.  Looking at the
> history for posixmodule.c etc will offer some clues as to how this could
> best be done.

One problem with removal, PyArg_ParseTuple doesn't have option to convert
to unicode (like "et" exists for unicode -> 8bit char buffer), so it's
harder to
report argument error.

>>> os.rename(4, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: rename() argument 1 must be string, not int

/* After removal of *A win32 APIs */

if (PyArg_ParseTuple("OO:rename", &o1, &o2)) {
    convert_to_unicode(&o1); /* if these methods failed, should we report
same error above
    convert_to_unicode(&o2);  * by ourselves? */


From solipsis at pitrou.net  Mon Apr 28 10:26:22 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Apr 2008 08:26:22 +0000 (UTC)
Subject: [Python-Dev] Dealing with a desired change to
	warnings.showwarning()
References: <bbaeab100804271340q21b4a364n41c8e4c62c8d3706@mail.gmail.com>
Message-ID: <loom.20080428T082157-602@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> As http://bugs.python.org/issue2705 points out, though, since the
> function has been documented as being allowed to be overridden, this
> potentially breaks existing showwarning() implementations. That means
> something needs to change.
> 
> One option is to introduce a new function. But what to name it?
> show_warning()? That small of a name change seems like asking for
> trouble, but showwarning_ex() just seems wrong to use in Python code.

But then what happens if a library overrides one of the callbacks and another
library overrides the other one? Fixing the original "problem" seems to
introduce other hairier ones.

As a Benjamin pointed out in the bug comments, the sane solution would be to
keep an unique callback and mention the small change in the release notes.

Regards

Antoine.



From ndbecker2 at gmail.com  Mon Apr 28 14:02:57 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 28 Apr 2008 08:02:57 -0400
Subject: [Python-Dev] enhanced ioctl?
Message-ID: <fv4edh$aph$1@ger.gmane.org>

IIUC, current ioctl is not capable of handling arbitrary argument types.
This code will allow any arg type (such as structures with pointers to
embedded structures).

The code for _IOC is taken from linux and might not be portable.

from ctypes import *

libc = CDLL ('/lib/libc.so.6')
#print libc.ioctl

def set_ioctl_argtype (arg_type):
    libc.ioctl.argtypes = (c_int, c_int, arg_type)

_IOC_WRITE = 0x1

_IOC_NRBITS=    8
_IOC_TYPEBITS=  8
_IOC_SIZEBITS=  14
_IOC_DIRBITS=   2

_IOC_NRSHIFT=   0
_IOC_TYPESHIFT= (_IOC_NRSHIFT+_IOC_NRBITS)
_IOC_SIZESHIFT= (_IOC_TYPESHIFT+_IOC_TYPEBITS)
_IOC_DIRSHIFT=  (_IOC_SIZESHIFT+_IOC_SIZEBITS)


def _IOC (dir, type, nr, size):
    return (((dir)  << _IOC_DIRSHIFT) | \
         ((type) << _IOC_TYPESHIFT) | \
         ((nr)   << _IOC_NRSHIFT) | \
         ((size) << _IOC_SIZESHIFT))

def ioctl (fd, request, args):
    return libc.ioctl (fd, request, args)
-------------------
Example usage:

from ioctl import *

set_ioctl_argtype (POINTER (eos_dl_args_t))

EOS_IOC_MAGIC = 0xF4
request = _IOC(_IOC_WRITE, EOS_IOC_MAGIC, 0x00, 0) # ignore size

err = ioctl (eos_file.fileno(), request, args)



From musiccomposition at gmail.com  Mon Apr 28 22:41:12 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 28 Apr 2008 15:41:12 -0500
Subject: [Python-Dev] enhanced ioctl?
In-Reply-To: <fv4edh$aph$1@ger.gmane.org>
References: <fv4edh$aph$1@ger.gmane.org>
Message-ID: <1afaf6160804281341w5fbd4b73j4b51487b7aff43a0@mail.gmail.com>

On Mon, Apr 28, 2008 at 7:02 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> IIUC, current ioctl is not capable of handling arbitrary argument types.
>  This code will allow any arg type (such as structures with pointers to
>  embedded structures).

Please submit this patch to the tracker.


-- 
Cheers,
Benjamin Peterson

From ndbecker2 at gmail.com  Mon Apr 28 23:05:11 2008
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 28 Apr 2008 17:05:11 -0400
Subject: [Python-Dev] enhanced ioctl?
References: <fv4edh$aph$1@ger.gmane.org>
	<1afaf6160804281341w5fbd4b73j4b51487b7aff43a0@mail.gmail.com>
Message-ID: <fv5e68$7s4$1@ger.gmane.org>

Benjamin Peterson wrote:

> On Mon, Apr 28, 2008 at 7:02 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> IIUC, current ioctl is not capable of handling arbitrary argument types.
>>  This code will allow any arg type (such as structures with pointers to
>>  embedded structures).
> 
> Please submit this patch to the tracker.
> 
> 

issue 2712


From guido at python.org  Tue Apr 29 01:01:09 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2008 16:01:09 -0700
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <18452.65322.843226.299769@montanaro-dyndns-org.local>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>

Have we documented the alternatives well enough? In most cases
NamedTemporaryFile will work, but sometimes you will have to create a
directory and pick names therein. Doing that so that it will always be
cleaned up properly is a bit of a trick, involving an isdir() check
and a shutil.rmtree() call.

On Sun, Apr 27, 2008 at 3:33 PM,  <skip at pobox.com> wrote:
> Back in r29829, Guido commented out the security hole warning for
>  tempfile.mktemp:
>
>     r29829 | gvanrossum | 2002-11-22 09:56:29 -0600 (Fri, 22 Nov 2002) | 3 lines
>
>     Comment out the warnings about mktemp().  These are too annoying, and
>     often unavoidable.
>
>  Any thought about whether this warning should be restored?  We're 5+ years
>  later.  Hopefully many uses of mktemp have been removed.  If we're not going
>  to restore the warning perhaps the commented code should just be deleted.
>
>  Skip
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Apr 29 01:17:54 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 28 Apr 2008 18:17:54 -0500
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
Message-ID: <18454.23330.147475.830232@montanaro-dyndns-org.local>


    Guido> Have we documented the alternatives well enough? 

I suppose we could document explicitly how to use mkstemp() in place of
mktemp(), but the difference in return value is fairly modest:

    >>> tempfile.mktemp()
    '/var/folders/5q/5qTPn6xq2RaWqk+1Ytw3-U+++TI/-Tmp-/tmpV_5OLi'
    >>> tempfile.mkstemp()
    (3, '/var/folders/5q/5qTPn6xq2RaWqk+1Ytw3-U+++TI/-Tmp-/tmpmS7K4T')

and the argument list is quite similar as well:

    >>> help(tempfile.mktemp)
    Help on function mktemp in module tempfile:

    mktemp(suffix='', prefix='tmp', dir=None)
    ...
    >>> help(tempfile.mkstemp) 
    Help on function mkstemp in module tempfile:

    mkstemp(suffix='', prefix='tmp', dir=None, text=False)
    ...

    Guido> In most cases NamedTemporaryFile will work, ...

It's API seems to be a bit farther from the mktemp API than that of mkstemp.

Skip

From guido at python.org  Tue Apr 29 01:22:42 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2008 16:22:42 -0700
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <18454.23330.147475.830232@montanaro-dyndns-org.local>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>

IMO mkstemp() is a major pain because you have to use raw file
descriptors on the return value. I'd much rather recommend
[Named]TemporaryFile which return streams.

On Mon, Apr 28, 2008 at 4:17 PM,  <skip at pobox.com> wrote:
>
>     Guido> Have we documented the alternatives well enough?
>
>  I suppose we could document explicitly how to use mkstemp() in place of
>  mktemp(), but the difference in return value is fairly modest:
>
>     >>> tempfile.mktemp()
>     '/var/folders/5q/5qTPn6xq2RaWqk+1Ytw3-U+++TI/-Tmp-/tmpV_5OLi'
>     >>> tempfile.mkstemp()
>     (3, '/var/folders/5q/5qTPn6xq2RaWqk+1Ytw3-U+++TI/-Tmp-/tmpmS7K4T')
>
>  and the argument list is quite similar as well:
>
>     >>> help(tempfile.mktemp)
>     Help on function mktemp in module tempfile:
>
>     mktemp(suffix='', prefix='tmp', dir=None)
>     ...
>     >>> help(tempfile.mkstemp)
>     Help on function mkstemp in module tempfile:
>
>     mkstemp(suffix='', prefix='tmp', dir=None, text=False)
>     ...
>
>     Guido> In most cases NamedTemporaryFile will work, ...
>
>  It's API seems to be a bit farther from the mktemp API than that of mkstemp.
>
>  Skip
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Apr 29 01:35:28 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 28 Apr 2008 18:35:28 -0500
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
Message-ID: <18454.24384.41749.272069@montanaro-dyndns-org.local>


    Guido> IMO mkstemp() is a major pain because you have to use raw file
    Guido> descriptors on the return value. I'd much rather recommend
    Guido> [Named]TemporaryFile which return streams.

Why not:

    fd, fname = tempfile.mkstemp()
    f = os.fdopen(fd)

Seems fairly straightforward to me.

Skip

From guido at python.org  Tue Apr 29 01:50:19 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Apr 2008 16:50:19 -0700
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <18454.24384.41749.272069@montanaro-dyndns-org.local>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<18454.24384.41749.272069@montanaro-dyndns-org.local>
Message-ID: <ca471dc20804281650s3501ddb8x572b09cff6a86278@mail.gmail.com>

I'd be much happier if there was a standard API in tempfile.py that
did that for you, so you wouldn't have to understand fdopen(). (Your
example is wrong BTW, the open mode would have to be something like
'rb+' or 'wb+'.)

On Mon, Apr 28, 2008 at 4:35 PM,  <skip at pobox.com> wrote:
>
>     Guido> IMO mkstemp() is a major pain because you have to use raw file
>     Guido> descriptors on the return value. I'd much rather recommend
>     Guido> [Named]TemporaryFile which return streams.
>
>  Why not:
>
>     fd, fname = tempfile.mkstemp()
>     f = os.fdopen(fd)
>
>  Seems fairly straightforward to me.
>
>  Skip
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Apr 29 04:12:21 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 28 Apr 2008 21:12:21 -0500
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <ca471dc20804281650s3501ddb8x572b09cff6a86278@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<18454.24384.41749.272069@montanaro-dyndns-org.local>
	<ca471dc20804281650s3501ddb8x572b09cff6a86278@mail.gmail.com>
Message-ID: <18454.33797.918894.424380@montanaro-dyndns-org.local>


    Guido> I'd be much happier if there was a standard API in tempfile.py
    Guido> that did that for you, so you wouldn't have to understand
    Guido> fdopen().

That can be arranged:

    http://bugs.python.org/issue2717

    Guido> (Your example is wrong BTW, the open mode would have to be
    Guido> something like 'rb+' or 'wb+'.)

I'm not disagreeing with you, but it seems odd that os.fdopen doesn't simply
obey the mode of the file descriptor it receives as its argument (e.g.,
normally opened with "w+b").

Skip


From greg.ewing at canterbury.ac.nz  Tue Apr 29 07:15:11 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Apr 2008 17:15:11 +1200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <18454.23330.147475.830232@montanaro-dyndns-org.local>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
Message-ID: <4816AEDF.1060905@canterbury.ac.nz>

skip at pobox.com wrote:
>     Guido> Have we documented the alternatives well enough? 
> 
> I suppose we could document explicitly how to use mkstemp() in place of
> mktemp(), but the difference in return value is fairly modest:

I'd like to see a variation of mkstemp() that returns
a file object instead of a file descriptor, since that's
what you really want most of the time. At least I always
end up calling fdopen on it.

-- 
Greg

From rasky at develer.com  Tue Apr 29 07:43:34 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 29 Apr 2008 05:43:34 +0000 (UTC)
Subject: [Python-Dev] Warn about mktemp once again?
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<4816AEDF.1060905@canterbury.ac.nz>
Message-ID: <fv6ci6$gja$1@ger.gmane.org>

On Tue, 29 Apr 2008 17:15:11 +1200, Greg Ewing wrote:

> skip at pobox.com wrote:
>>     Guido> Have we documented the alternatives well enough?
>> 
>> I suppose we could document explicitly how to use mkstemp() in place of
>> mktemp(), but the difference in return value is fairly modest:
> 
> I'd like to see a variation of mkstemp() that returns a file object
> instead of a file descriptor, since that's what you really want most of
> the time. At least I always end up calling fdopen on it.

Same here. In fact, is there a good reason to have mkstemp() return the 
fd (except backward compatibility)?
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From greg.ewing at canterbury.ac.nz  Tue Apr 29 07:49:33 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Apr 2008 17:49:33 +1200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
Message-ID: <4816B6ED.6030501@canterbury.ac.nz>

Guido van Rossum wrote:
> IMO mkstemp() is a major pain because you have to use raw file
> descriptors on the return value. I'd much rather recommend
> [Named]TemporaryFile which return streams.

The problem with NamedTemporaryFile is that it deletes
the file as soon as you close it, which makes the
named-ness of it rather useless, as far as I can see.
If you don't want that to happen, you have to use
mkstemp.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Apr 29 07:50:33 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Apr 2008 17:50:33 +1200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <18454.33797.918894.424380@montanaro-dyndns-org.local>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<18454.24384.41749.272069@montanaro-dyndns-org.local>
	<ca471dc20804281650s3501ddb8x572b09cff6a86278@mail.gmail.com>
	<18454.33797.918894.424380@montanaro-dyndns-org.local>
Message-ID: <4816B729.9090108@canterbury.ac.nz>

skip at pobox.com wrote:
> I'm not disagreeing with you, but it seems odd that os.fdopen doesn't simply
> obey the mode of the file descriptor it receives as its argument

I'm not even sure if it's possible to find out the mode
that a file descriptor was opened with -- is it?

Certainly there's no way to tell whether you want it
treated as binary, so at least that much of the mode
needs to be specified.

-- 
Greg

From scott+python-dev at scottdial.com  Tue Apr 29 08:05:49 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 29 Apr 2008 02:05:49 -0400
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <4816B729.9090108@canterbury.ac.nz>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>	<18454.24384.41749.272069@montanaro-dyndns-org.local>	<ca471dc20804281650s3501ddb8x572b09cff6a86278@mail.gmail.com>	<18454.33797.918894.424380@montanaro-dyndns-org.local>
	<4816B729.9090108@canterbury.ac.nz>
Message-ID: <4816BABD.9080708@scottdial.com>

Greg Ewing wrote:
> skip at pobox.com wrote:
>> I'm not disagreeing with you, but it seems odd that os.fdopen doesn't 
>> simply
>> obey the mode of the file descriptor it receives as its argument
> 
> I'm not even sure if it's possible to find out the mode
> that a file descriptor was opened with -- is it?
> 

int mode = fcntl(fd, F_GETFL);
switch(mode) {
case O_RDONLY: ...
case O_WRONLY: ...
case O_RDWR: ...
}

> Certainly there's no way to tell whether you want it
> treated as binary, so at least that much of the mode
> needs to be specified.

Agreed.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From martin at v.loewis.de  Tue Apr 29 08:35:00 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Apr 2008 08:35:00 +0200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <fv6ci6$gja$1@ger.gmane.org>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<4816AEDF.1060905@canterbury.ac.nz>
	<fv6ci6$gja$1@ger.gmane.org>
Message-ID: <4816C194.9060103@v.loewis.de>

> Same here. In fact, is there a good reason to have mkstemp() return the 
> fd (except backward compatibility)?

Except for backwards compatibility: is there a good reason to keep
os.mkstemp at all?

The tradition is that all APIs in os expose the "system" calls as-is,
i.e. they don't try to adjust the result values at all. This has
the advantage that the caller can use their system's man page to
find out all details, and trust that the Python wrapper does exactly
the same.

For mkstemp, there are two special issues: the C API modifies the
string parameter, which cannot be exposed to Python as-is (hence
the two result values), and it isn't a system call (at least not
on POSIX), so it doesn't really need to be exposed, except perhaps
to also provide this on non-POSIX systems where a regular POSIX
implementation (in Python) would fail.

Regards,
Martin


From dave.l.harrison at gmail.com  Tue Apr 29 11:58:39 2008
From: dave.l.harrison at gmail.com (David Harrison)
Date: Tue, 29 Apr 2008 19:58:39 +1000
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <4816C194.9060103@v.loewis.de>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<4816AEDF.1060905@canterbury.ac.nz> <fv6ci6$gja$1@ger.gmane.org>
	<4816C194.9060103@v.loewis.de>
Message-ID: <a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>

2008/4/29 "Martin v. L?wis" <martin at v.loewis.de>:
> > Same here. In fact, is there a good reason to have mkstemp() return the
>  > fd (except backward compatibility)?
>
>  Except for backwards compatibility: is there a good reason to keep
>  os.mkstemp at all?

Greg Ewing's use-case is one I've also had at times - ie. as a
convenience function for creating a "somewhat temporary" file that is
randomly named, but persists beyond the closing of the file.  If the
function doesn't stay in os it doesn't make any difference to me
though :-)

From skip at pobox.com  Tue Apr 29 12:55:10 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 29 Apr 2008 05:55:10 -0500
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <4816AEDF.1060905@canterbury.ac.nz>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<4816AEDF.1060905@canterbury.ac.nz>
Message-ID: <18454.65166.166747.638082@montanaro-dyndns-org.local>


    Greg> I'd like to see a variation of mkstemp() that returns a file
    Greg> object instead of a file descriptor...

http://bugs.python.org/issue2717

Comments welcome.

Skip

From ncoghlan at gmail.com  Tue Apr 29 14:15:00 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 29 Apr 2008 22:15:00 +1000
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <4816B6ED.6030501@canterbury.ac.nz>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<4816B6ED.6030501@canterbury.ac.nz>
Message-ID: <48171144.6050706@gmail.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
>> IMO mkstemp() is a major pain because you have to use raw file
>> descriptors on the return value. I'd much rather recommend
>> [Named]TemporaryFile which return streams.
> 
> The problem with NamedTemporaryFile is that it deletes
> the file as soon as you close it, which makes the
> named-ness of it rather useless, as far as I can see.
> If you don't want that to happen, you have to use
> mkstemp.

That much has been fixed in 2.6 - you can now just pass "delete=False" 
to the NamedTemporaryFile constructor to stop it doing that.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Tue Apr 29 14:18:16 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 29 Apr 2008 22:18:16 +1000
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<4816AEDF.1060905@canterbury.ac.nz>
	<fv6ci6$gja$1@ger.gmane.org>	<4816C194.9060103@v.loewis.de>
	<a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
Message-ID: <48171208.9030604@gmail.com>

David Harrison wrote:
> 2008/4/29 "Martin v. L?wis" <martin at v.loewis.de>:
>>> Same here. In fact, is there a good reason to have mkstemp() return the
>>  > fd (except backward compatibility)?
>>
>>  Except for backwards compatibility: is there a good reason to keep
>>  os.mkstemp at all?
> 
> Greg Ewing's use-case is one I've also had at times - ie. as a
> convenience function for creating a "somewhat temporary" file that is
> randomly named, but persists beyond the closing of the file.  If the
> function doesn't stay in os it doesn't make any difference to me
> though :-)

As of 2.6, Greg's use case is addressed by the new 'delete' parameter on 
tempfile.NamedTemporaryFile.

The implementation of the tempfile module uses os.mkstemp() though, so 
getting rid of the latter might cause a few problems :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From rasky at develer.com  Tue Apr 29 15:34:24 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 29 Apr 2008 15:34:24 +0200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <48171208.9030604@gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<4816AEDF.1060905@canterbury.ac.nz>
	<fv6ci6$gja$1@ger.gmane.org>	<4816C194.9060103@v.loewis.de>
	<a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
	<48171208.9030604@gmail.com>
Message-ID: <481723E0.8020608@develer.com>

On 4/29/2008 2:18 PM, Nick Coghlan wrote:

>>>> Same here. In fact, is there a good reason to have mkstemp() return the
>>>  > fd (except backward compatibility)?
>>>
>>>  Except for backwards compatibility: is there a good reason to keep
>>>  os.mkstemp at all?
>>
>> Greg Ewing's use-case is one I've also had at times - ie. as a
>> convenience function for creating a "somewhat temporary" file that is
>> randomly named, but persists beyond the closing of the file.  If the
>> function doesn't stay in os it doesn't make any difference to me
>> though :-)
> 
> As of 2.6, Greg's use case is addressed by the new 'delete' parameter on 
> tempfile.NamedTemporaryFile.

Then I personally don't have any objection to the removal of os.mkstemp.

Since we're at it, a common pattern I use is to create temporary file to 
atomically replace files: I create a named temporary file in the same 
directory of the file I want to replace; write data into it; close it; 
and move it (POSIX "move": rename with silent overwrite) to the 
destination file. AFAIK, this is allows an atomic file replacemente on 
most filesystems.

I believe this is a common useful pattern that could be handled in 
module tmpfile (especially since the final "rename" step requires a 
little care to be truly multiplatform).
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

From tnelson at onresolve.com  Tue Apr 29 15:58:02 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 29 Apr 2008 06:58:02 -0700
Subject: [Python-Dev] socket.try_reuse_address()
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>


Since the recent changes to networking-oriented tests (issue 2550, r62234 and r62237), I think it's safe to say stability of the test suite on all the non-Windows platforms has improved significantly in this area (I don't recall seeing any socket errors in *nix buildbot logs since those commits).

However, Windows buildbots are still periodically failing.  More specifically, my Windows buildbots are still failing.  One thing that's different about my buildbots is that two are being run at the same time for both trunk and py3k -- one doing an x86 build, the other doing x64 build.

Since the changes in the aforementioned revisions, the behaviour of my buildbots has definitely improved -- they no longer completely wedge on test_async(chat|core), mainly due to abolishing all use of SO_REUSEADDR as a socket option in any network-oriented tests.

However, periodically, they're still dying/failing in a variety of ways -- see relevant log snippets at the end of this e-mail for examples.  I attribute this to the fact that SO_REUSEADDR is still set as a socket option in asyncore.py and SocketServer.py.  Basically, SO_REUSEADDR should *never* be set on Windows for TCP/IP sockets.  Using asyncore.py as an example, here are two ways we could handle this:

1. Hard code the Windows opt-out:
--- asyncore.py (revision 62509)
+++ asyncore.py (working copy)
@@ -267,6 +267,8 @@

     def set_reuse_addr(self):
         # try to re-use a server port if possible
+        if os.name == 'nt' and self.socket.socket_type != socket.SOCK_DGRAM:
+            return
         try:
             self.socket.setsockopt(
                 socket.SOL_SOCKET, socket.SO_REUSEADDR,

2. Introduce socket.try_reuse_address():
--- asyncore.py (revision 62509)
+++ asyncore.py (working copy)
@@ -266,15 +266,7 @@
         self.add_channel(map)

     def set_reuse_addr(self):
-        # try to re-use a server port if possible
-        try:
-            self.socket.setsockopt(
-                socket.SOL_SOCKET, socket.SO_REUSEADDR,
-                self.socket.getsockopt(socket.SOL_SOCKET,
-                                       socket.SO_REUSEADDR) | 1
-                )
-        except socket.error:
-            pass
+        self.socket.try_reuse_address()


With try_use_address implemented as follows:

--- socket.py   (revision 62509)
+++ socket.py   (working copy)
@@ -197,6 +197,10 @@
         Return a new socket object connected to the same system resource."""
         return _socketobject(_sock=self._sock)

+    def try_reuse_address(self):
+        if not (os.name == 'nt' and self._sock.type != SOCK_DGRAM):
+            self._sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
+
     def makefile(self, mode='r', bufsize=-1):
         """makefile([mode[, bufsize]]) -> file object

I prefer the latter as it's cleaner, easier to document and encapsulates what we're trying to do relatively well.  The affected modules would be asyncore.py, SocketServer.py and idlelib/rpc.py.  Thoughts?

Regards,

        Trent.


<eg1>
test_ftplib

remoteFailed: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.
]
</eg1>

<eg2>
test_asynchat
test test_asynchat failed -- errors occurred; run in verbose mode for details
[snip to bottom of log where test_asynchat is re-run]
1 test failed:
    test_asynchat
33 tests skipped:
    test__locale test_aepack test_applesingle test_cProfile
    test_commands test_crypt test_curses test_dbm test_epoll
    test_fcntl test_fork1 test_gdbm test_grp test_ioctl test_kqueue
    test_macostools test_mhlib test_nis test_openpty test_ossaudiodev
    test_pipes test_poll test_posix test_pty test_pwd test_resource
    test_scriptpackages test_signal test_syslog test_threadsignals
    test_wait3 test_wait4 test_zipfile64
Those skips are all expected on win32.
Re-running failed tests in verbose mode
Re-running test 'test_asynchat' in verbose mode
test_close_when_done (test.test_asynchat.TestAsynchat) ... ok
test_empty_line (test.test_asynchat.TestAsynchat) ... ok
test_line_terminator1 (test.test_asynchat.TestAsynchat) ... ok
test_line_terminator2 (test.test_asynchat.TestAsynchat) ... ok
test_line_terminator3 (test.test_asynchat.TestAsynchat) ... ok
test_none_terminator (test.test_asynchat.TestAsynchat) ... ok
test_numeric_terminator1 (test.test_asynchat.TestAsynchat) ... ok
test_numeric_terminator2 (test.test_asynchat.TestAsynchat) ... ok
test_simple_producer (test.test_asynchat.TestAsynchat) ... ok
test_string_producer (test.test_asynchat.TestAsynchat) ... ok
test_close_when_done (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_empty_line (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_line_terminator1 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_line_terminator2 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_line_terminator3 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_none_terminator (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_numeric_terminator1 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_numeric_terminator2 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_simple_producer (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_string_producer (test.test_asynchat.TestAsynchat_WithPoll) ... ok
test_find_prefix_at_end (test.test_asynchat.TestHelperFunctions) ... ok
test_basic (test.test_asynchat.TestFifo) ... ok
test_given_list (test.test_asynchat.TestFifo) ... ok

----------------------------------------------------------------------
Ran 23 tests in 11.812s

OK
</eg2>
(Note that re-running the test here didn't result in the test failing again.)

<eg3>
1 test failed:
    test_smtplib

Traceback (most recent call last):
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\threading.py", line 493, in _bootstrap_inner
    self.run()
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\threading.py", line 449, in run
    self._target(*self._args, **self._kwargs)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\test\test_smtplib.py", line 106, in debugging_server
    poll_fun(0.01, asyncore.socket_map)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 132, in poll
    read(obj)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 72, in read
    obj.handle_error()
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 68, in read
    obj.handle_read_event()
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 390, in handle_read_event
    self.handle_read()
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\test\test_ssl.py", line 524, in handle_read
    data = self.recv(1024)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 342, in recv
    data = self.socket.recv(buffer_size)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\ssl.py", line 247, in recv
    return self.read(buflen)
  File "S:\buildbots\python\3.0.nelson-windows\build\lib\ssl.py", line 162, in read
    v = self._sslobj.read(len or 1024)
socket.error: [Errno 10053] An established connection was aborted by the software in your host machine
</eg3>



From hrvoje.niksic at avl.com  Tue Apr 29 16:16:06 2008
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Tue, 29 Apr 2008 16:16:06 +0200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <481723E0.8020608@develer.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<4816AEDF.1060905@canterbury.ac.nz> <fv6ci6$gja$1@ger.gmane.org>
	<4816C194.9060103@v.loewis.de>
	<a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
	<48171208.9030604@gmail.com>  <481723E0.8020608@develer.com>
Message-ID: <1209478566.7635.4.camel@localhost>

On Tue, 2008-04-29 at 15:34 +0200, Giovanni Bajo wrote:
> > As of 2.6, Greg's use case is addressed by the new 'delete' parameter on 
> > tempfile.NamedTemporaryFile.
> 
> Then I personally don't have any objection to the removal of os.mkstemp.

os.mkstemp is still useful when you *don't* need the file object, but
the actual file descriptor, for passing to C code or to the child
processes, or to mmap.mmap and such.  It is also familiar to the Unix/C
hackers, and it should cost very little to keep it around.

> Since we're at it, a common pattern I use is to create temporary file to 
> atomically replace files: I create a named temporary file in the same 
> directory of the file I want to replace; write data into it; close it; 
> and move it (POSIX "move": rename with silent overwrite) to the 
> destination file. AFAIK, this is allows an atomic file replacemente on 
> most filesystems.
> 
> I believe this is a common useful pattern that could be handled in 
> module tmpfile (especially since the final "rename" step requires a 
> little care to be truly multiplatform).

I agree, having a tempfile class with rename-on-close semantics would be
very useful.


From gnewsg at gmail.com  Tue Apr 29 16:43:04 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Tue, 29 Apr 2008 07:43:04 -0700 (PDT)
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
Message-ID: <b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>

Maybe it would be better considering Windows CE systems too:

- if os.name == 'nt'
+ if os.name in ('nt', 'ce')

Moreover if the method is called "try_reuse_address" I'd expect that
"self._sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)" would be placed
inside a try/except block.

On 29 Apr, 15:58, Trent Nelson <tnel... at onresolve.com> wrote:
> Since the recent changes to networking-oriented tests (issue 2550, r62234 and r62237), I think it's safe to say stability of the test suite on all the non-Windows platforms has improved significantly in this area (I don't recall seeing any socket errors in *nix buildbot logs since those commits).
>
> However, Windows buildbots are still periodically failing. ?More specifically, my Windows buildbots are still failing. ?One thing that's different about my buildbots is that two are being run at the same time for both trunk and py3k -- one doing an x86 build, the other doing x64 build.
>
> Since the changes in the aforementioned revisions, the behaviour of my buildbots has definitely improved -- they no longer completely wedge on test_async(chat|core), mainly due to abolishing all use of SO_REUSEADDR as a socket option in any network-oriented tests.
>
> However, periodically, they're still dying/failing in a variety of ways -- see relevant log snippets at the end of this e-mail for examples. ?I attribute this to the fact that SO_REUSEADDR is still set as a socket option in asyncore.py and SocketServer.py. ?Basically, SO_REUSEADDR should *never* be set on Windows for TCP/IP sockets. ?Using asyncore.py as an example, here are two ways we could handle this:
>
> 1. Hard code the Windows opt-out:
> --- asyncore.py (revision 62509)
> +++ asyncore.py (working copy)
> @@ -267,6 +267,8 @@
>
> ? ? ?def set_reuse_addr(self):
> ? ? ? ? ?# try to re-use a server port if possible
> + ? ? ? ?if os.name == 'nt' and self.socket.socket_type != socket.SOCK_DGRAM:
> + ? ? ? ? ? ?return
> ? ? ? ? ?try:
> ? ? ? ? ? ? ?self.socket.setsockopt(
> ? ? ? ? ? ? ? ? ?socket.SOL_SOCKET, socket.SO_REUSEADDR,
>
> 2. Introduce socket.try_reuse_address():
> --- asyncore.py (revision 62509)
> +++ asyncore.py (working copy)
> @@ -266,15 +266,7 @@
> ? ? ? ? ?self.add_channel(map)
>
> ? ? ?def set_reuse_addr(self):
> - ? ? ? ?# try to re-use a server port if possible
> - ? ? ? ?try:
> - ? ? ? ? ? ?self.socket.setsockopt(
> - ? ? ? ? ? ? ? ?socket.SOL_SOCKET, socket.SO_REUSEADDR,
> - ? ? ? ? ? ? ? ?self.socket.getsockopt(socket.SOL_SOCKET,
> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? socket.SO_REUSEADDR) | 1
> - ? ? ? ? ? ? ? ?)
> - ? ? ? ?except socket.error:
> - ? ? ? ? ? ?pass
> + ? ? ? ?self.socket.try_reuse_address()
>
> With try_use_address implemented as follows:
>
> --- socket.py ? (revision 62509)
> +++ socket.py ? (working copy)
> @@ -197,6 +197,10 @@
> ? ? ? ? ?Return a new socket object connected to the same system resource."""
> ? ? ? ? ?return _socketobject(_sock=self._sock)
>
> + ? ?def try_reuse_address(self):
> + ? ? ? ?if not (os.name == 'nt' and self._sock.type != SOCK_DGRAM):
> + ? ? ? ? ? ?self._sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
> +
> ? ? ?def makefile(self, mode='r', bufsize=-1):
> ? ? ? ? ?"""makefile([mode[, bufsize]]) -> file object
>
> I prefer the latter as it's cleaner, easier to document and encapsulates what we're trying to do relatively well. ?The affected modules would be asyncore.py, SocketServer.py and idlelib/rpc.py. ?Thoughts?
>
> Regards,
>
> ? ? ? ? Trent.
>
> <eg1>
> test_ftplib
>
> remoteFailed: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.
> ]
> </eg1>
>
> <eg2>
> test_asynchat
> test test_asynchat failed -- errors occurred; run in verbose mode for details
> [snip to bottom of log where test_asynchat is re-run]
> 1 test failed:
> ? ? test_asynchat
> 33 tests skipped:
> ? ? test__locale test_aepack test_applesingle test_cProfile
> ? ? test_commands test_crypt test_curses test_dbm test_epoll
> ? ? test_fcntl test_fork1 test_gdbm test_grp test_ioctl test_kqueue
> ? ? test_macostools test_mhlib test_nis test_openpty test_ossaudiodev
> ? ? test_pipes test_poll test_posix test_pty test_pwd test_resource
> ? ? test_scriptpackages test_signal test_syslog test_threadsignals
> ? ? test_wait3 test_wait4 test_zipfile64
> Those skips are all expected on win32.
> Re-running failed tests in verbose mode
> Re-running test 'test_asynchat' in verbose mode
> test_close_when_done (test.test_asynchat.TestAsynchat) ... ok
> test_empty_line (test.test_asynchat.TestAsynchat) ... ok
> test_line_terminator1 (test.test_asynchat.TestAsynchat) ... ok
> test_line_terminator2 (test.test_asynchat.TestAsynchat) ... ok
> test_line_terminator3 (test.test_asynchat.TestAsynchat) ... ok
> test_none_terminator (test.test_asynchat.TestAsynchat) ... ok
> test_numeric_terminator1 (test.test_asynchat.TestAsynchat) ... ok
> test_numeric_terminator2 (test.test_asynchat.TestAsynchat) ... ok
> test_simple_producer (test.test_asynchat.TestAsynchat) ... ok
> test_string_producer (test.test_asynchat.TestAsynchat) ... ok
> test_close_when_done (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_empty_line (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_line_terminator1 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_line_terminator2 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_line_terminator3 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_none_terminator (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_numeric_terminator1 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_numeric_terminator2 (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_simple_producer (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_string_producer (test.test_asynchat.TestAsynchat_WithPoll) ... ok
> test_find_prefix_at_end (test.test_asynchat.TestHelperFunctions) ... ok
> test_basic (test.test_asynchat.TestFifo) ... ok
> test_given_list (test.test_asynchat.TestFifo) ... ok
>
> ----------------------------------------------------------------------
> Ran 23 tests in 11.812s
>
> OK
> </eg2>
> (Note that re-running the test here didn't result in the test failing again.)
>
> <eg3>
> 1 test failed:
> ? ? test_smtplib
>
> Traceback (most recent call last):
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\threading.py", line 493, in _bootstrap_inner
> ? ? self.run()
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\threading.py", line 449, in run
> ? ? self._target(*self._args, **self._kwargs)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\test\test_smtplib.py", line 106, in debugging_server
> ? ? poll_fun(0.01, asyncore.socket_map)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 132, in poll
> ? ? read(obj)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 72, in read
> ? ? obj.handle_error()
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 68, in read
> ? ? obj.handle_read_event()
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 390, in handle_read_event
> ? ? self.handle_read()
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\test\test_ssl.py", line 524, in handle_read
> ? ? data = self.recv(1024)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\asyncore.py", line 342, in recv
> ? ? data = self.socket.recv(buffer_size)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\ssl.py", line 247, in recv
> ? ? return self.read(buflen)
> ? File "S:\buildbots\python\3.0.nelson-windows\build\lib\ssl.py", line 162, in read
> ? ? v = self._sslobj.read(len or 1024)
> socket.error: [Errno 10053] An established connection was aborted by the software in your host machine
> </eg3>
>
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From steve at holdenweb.com  Tue Apr 29 17:20:19 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 29 Apr 2008 11:20:19 -0400
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
Message-ID: <fv7ecd$8da$1@ger.gmane.org>

Giampaolo Rodola' wrote:
> Maybe it would be better considering Windows CE systems too:
> 
> - if os.name == 'nt'
> + if os.name in ('nt', 'ce')
> 
Cygwin? I don't know how Unix-like it is.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From tnelson at onresolve.com  Tue Apr 29 18:27:43 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Tue, 29 Apr 2008 09:27:43 -0700
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <fv7ecd$8da$1@ger.gmane.org>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477EF81@EXMBX04.exchhosting.com>


> Giampaolo Rodola' wrote:
> > Maybe it would be better considering Windows CE systems too:
> >
> > - if os.name == 'nt'
> > + if os.name in ('nt', 'ce')
> >
> Cygwin? I don't know how Unix-like it is.

Yeah, that's a fair point, it's the behaviour of the underlying Winsock API we're targeting, so it would apply to Cygwin as well.  (And CE and anything else on Windows.)

        Trent.

From guido at python.org  Tue Apr 29 21:09:02 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Apr 2008 12:09:02 -0700
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <4816B6ED.6030501@canterbury.ac.nz>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<4816B6ED.6030501@canterbury.ac.nz>
Message-ID: <ca471dc20804291209u32458a8wc4556aa19fecfd21@mail.gmail.com>

On Mon, Apr 28, 2008 at 10:49 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
>  The problem with NamedTemporaryFile is that it deletes
>  the file as soon as you close it, which makes the
>  named-ness of it rather useless, as far as I can see.

Why? You can flush it and then all the data is on the disk.

The whole point of [Named]TemporaryFile is to automate the cleanup as
well as the creation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From armin.ronacher at active-4.com  Tue Apr 29 21:30:07 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Tue, 29 Apr 2008 19:30:07 +0000 (UTC)
Subject: [Python-Dev] Module Suggestion: ast
Message-ID: <loom.20080429T192928-902@post.gmane.org>

Hi all,

I would like to propose a new module for the stdlib for Python 2.6
and higher:  "ast".  The motivation for this module is the pending
deprecation for compiler.ast which is widely used (debugging,
template engines, code coverage etc.).  _ast is a very solid module
and is without a doubt easier to maintain then compiler.ast which
was written in Python, it's lacking some features such as pretty
printing the AST or traversing it.

The idea of "ast" would be adding high level functionality for
easier working with the AST.  It currently provides these features:

-   pretty printing AST objects
-   a parse function as easier alias for compile() + flag
-   operator-node -> operator symbol mappings (eg: _ast.Add -> '+')
-   methods to modify lineno / col_offset (incrementing or copying
    the data over from existing nodes)
-   getting the fields of nodes as dicts
-   iterating over all child nodes
-   a function to get the docstring or an AST node
-   a node walker that yields all child-nodes recursively
-   a `NodeVistor` and `NodeTransformer`

Additionally there is a `literate_eval` function in that module that
can safely evaluate python literals in a string.

Module and unittests are located in the Pocoo Sandbox HG repository:

  http://dev.pocoo.org/hg/sandbox/file/tip/ast/ast.py
  http://dev.pocoo.org/hg/sandbox/file/tip/ast/test_ast.py

A slightly modified version of the ast.py module for Python 2.5
compatibility is currently in use for the Mako template engine to
achieve support for Google's AppEngine.

An example module for the NodeVisitor is in the repository which
converts a Python AST back into Python source code:

  http://dev.pocoo.org/hg/sandbox/file/tip/ast/codegen.py


Regards,
Armin


From martin at v.loewis.de  Tue Apr 29 23:35:17 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Apr 2008 23:35:17 +0200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>	<18454.23330.147475.830232@montanaro-dyndns-org.local>	<4816AEDF.1060905@canterbury.ac.nz>
	<fv6ci6$gja$1@ger.gmane.org>	<4816C194.9060103@v.loewis.de>
	<a1581f70804290258n9c3a6aek6add74e7d0c13bae@mail.gmail.com>
Message-ID: <48179495.8080808@v.loewis.de>

>>> Same here. In fact, is there a good reason to have mkstemp() return the
>>  > fd (except backward compatibility)?
>>
>>  Except for backwards compatibility: is there a good reason to keep
>>  os.mkstemp at all?
> 
> Greg Ewing's use-case is one I've also had at times - ie. as a
> convenience function for creating a "somewhat temporary" file that is
> randomly named, but persists beyond the closing of the file.  If the
> function doesn't stay in os it doesn't make any difference to me
> though :-)

I was talking about the specific implementation, which happens to be
a wrapper around the C library's mkstemp. For the use case in
question, I think passing delete=False to NamedTemporaryFile would
work just as well.

Regards,
Martin

From martin at v.loewis.de  Tue Apr 29 23:38:50 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Apr 2008 23:38:50 +0200
Subject: [Python-Dev] Module Suggestion: ast
In-Reply-To: <loom.20080429T192928-902@post.gmane.org>
References: <loom.20080429T192928-902@post.gmane.org>
Message-ID: <4817956A.6090507@v.loewis.de>

> An example module for the NodeVisitor is in the repository which
> converts a Python AST back into Python source code:
> 
>   http://dev.pocoo.org/hg/sandbox/file/tip/ast/codegen.py

And another example for the same functionality is in Demo/parser/unparse.py.

Regards,
Martin

From tom at vector-seven.com  Wed Apr 30 01:22:27 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Wed, 30 Apr 2008 09:22:27 +1000
Subject: [Python-Dev] Module Suggestion: ast
In-Reply-To: <loom.20080429T192928-902@post.gmane.org>
References: <loom.20080429T192928-902@post.gmane.org>
Message-ID: <4817ADB3.5000003@vector-seven.com>

Just a thought, but it would be great if this could be implemented over 
the top of a C layer that operates on real AST nodes (rather than the 
PyObject representation of those nodes). I'm working on stuff to perform 
code optimization at the AST level (see the tlee-ast-optimize branch), 
and the functionality you're describing may wind up being very useful to me.

I've got more to say on the topic, but I'm at work right now. Just 
something to keep in mind.

Cheers,
T

Armin Ronacher wrote:
> Hi all,
>
> I would like to propose a new module for the stdlib for Python 2.6
> and higher:  "ast".  The motivation for this module is the pending
> deprecation for compiler.ast which is widely used (debugging,
> template engines, code coverage etc.).  _ast is a very solid module
> and is without a doubt easier to maintain then compiler.ast which
> was written in Python, it's lacking some features such as pretty
> printing the AST or traversing it.
>
> The idea of "ast" would be adding high level functionality for
> easier working with the AST.  It currently provides these features:
>
> -   pretty printing AST objects
> -   a parse function as easier alias for compile() + flag
> -   operator-node -> operator symbol mappings (eg: _ast.Add -> '+')
> -   methods to modify lineno / col_offset (incrementing or copying
>     the data over from existing nodes)
> -   getting the fields of nodes as dicts
> -   iterating over all child nodes
> -   a function to get the docstring or an AST node
> -   a node walker that yields all child-nodes recursively
> -   a `NodeVistor` and `NodeTransformer`
>
> Additionally there is a `literate_eval` function in that module that
> can safely evaluate python literals in a string.
>
> Module and unittests are located in the Pocoo Sandbox HG repository:
>
>   http://dev.pocoo.org/hg/sandbox/file/tip/ast/ast.py
>   http://dev.pocoo.org/hg/sandbox/file/tip/ast/test_ast.py
>
> A slightly modified version of the ast.py module for Python 2.5
> compatibility is currently in use for the Mako template engine to
> achieve support for Google's AppEngine.
>
> An example module for the NodeVisitor is in the repository which
> converts a Python AST back into Python source code:
>
>   http://dev.pocoo.org/hg/sandbox/file/tip/ast/codegen.py
>
>
> Regards,
> Armin
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
>   


From greg.ewing at canterbury.ac.nz  Wed Apr 30 01:42:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Apr 2008 11:42:54 +1200
Subject: [Python-Dev] Warn about mktemp once again?
In-Reply-To: <ca471dc20804291209u32458a8wc4556aa19fecfd21@mail.gmail.com>
References: <18452.65322.843226.299769@montanaro-dyndns-org.local>
	<ca471dc20804281601w253e4778u56ac591b56427882@mail.gmail.com>
	<18454.23330.147475.830232@montanaro-dyndns-org.local>
	<ca471dc20804281622i1e7db186p39869661180ab7de@mail.gmail.com>
	<4816B6ED.6030501@canterbury.ac.nz>
	<ca471dc20804291209u32458a8wc4556aa19fecfd21@mail.gmail.com>
Message-ID: <4817B27E.8060704@canterbury.ac.nz>

Guido van Rossum wrote:

> Why? You can flush it and then all the data is on the disk.

That might be all right on Unix, but I would be
worried that having the file open could prevent
some other things being done with it on some
platforms, such as renaming. You might also want
to pass the file name on to some other process.

> The whole point of [Named]TemporaryFile is to automate the cleanup as
> well as the creation.

The docs (at least up to 2.5) don't make it at all
clear that this is the *whole* point of *both*
these functions.

The doc for NamedTemporaryFile seems to disavow
any guarantee that you will be able to do anything
with the name while the file is open. If you can't
use the name while the file is open, and the file
ceases to exist when it's closed, then what use is
it to have the name?

The obvious conclusion is that the point of
NamedTemporaryFile is to give you a file that *doesn't*
go away when you close it, and has a name so you can
go on to do other things with it.

So I plead ignorance due to misleading documentation.

-- 
Greg


From tom at vector-seven.com  Wed Apr 30 12:15:46 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Wed, 30 Apr 2008 20:15:46 +1000
Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with
	constant values?
Message-ID: <481846D2.6090509@vector-seven.com>

Hi all,

I've been working on optimization of the AST, including the porting of 
the old bytecode-level optimizations to the AST level. A few questions 
have come up in the process of doing this, all of which are probably 
appropriate for discussion on this list. The code I'm referring to here 
can be found in the tlee-ast-optimize branch. Most of the relevant code 
is in Python/optimize.c and Python/peephole.c.

Nearly all of the bytecode-level optimizations have been moved to the 
AST optimizer with a few exceptions. Most of those waiting to be ported 
are stuck in limbo due to the fact we can't yet inject arbitrary 
PyObject constants into the AST. Examples are tuples of constants and 
the optimization of "LOAD_GLOBAL/LOAD_NAME None" as "LOAD_CONST None".

This leaves us with a few options:

1. A new AST expr node for constant values for types other than Str/Num

I imagine this to be something like Const(PyObject* v), which is 
effectively translated to a "LOAD_CONST v" by the compiler. This trades 
the purity of the AST for a little practicality. A "Const" node has no 
real source representation, it would exist solely for the purpose of 
injecting PyObject constants into the AST.

2. Allow arbitrary annotations to be applied to the AST as compiler hints.

For example, each AST node might have an optional dict that contains a 
set of annotation values. Then when traversing the AST, the compiler 
might do something along the lines of:

if (expr->annotations) {
  PyObject* constvalue = PyDict_GetItemString(expr->annotations, 
"constantvalue");
  if (constvalue)
    ADDOP_O(c, LOAD_CONST, constvalue, consts)
  else
    VISIT(c, expr, expr)
}

This is a more general solution if we want to keep other compiler hints 
down the track, but unless somebody can think of another use case this 
is probably overkill.

3. Keep these particular optimizations at the bytecode level.

It would be great to be able to perform the optimizations at a higher 
level, but this would require no effort at all. This would mean two 
passes over the same code at two different levels.

If anybody familiar with this stuff could weigh in on the matter, it 
would be much appreciated. I've got a list of other issues that I need 
to discuss here, but this would be a great start.

Thanks,
Tom

From flashk at gmail.com  Wed Apr 30 18:36:44 2008
From: flashk at gmail.com (Farshid Lashkari)
Date: Wed, 30 Apr 2008 09:36:44 -0700
Subject: [Python-Dev] cStringIO buffer interface
Message-ID: <978d1eac0804300936j707324afude1c18ce12dba71f@mail.gmail.com>

Hello,

I'm not sure if this is the right place to ask, so I apologize ahead
of time if it is.

I was just curious as to why cStringIO objects don't implement the
buffer interface.  cStringIO objects seem similar to string and array
objects, and those support the buffer protocol. Is there a reason
against allowing cStringIO to support at least the read buffer
interface, or is just that nobody has considered it until now?

Thanks,
Farshid

From guido at python.org  Wed Apr 30 18:51:25 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Apr 2008 09:51:25 -0700
Subject: [Python-Dev] cStringIO buffer interface
In-Reply-To: <978d1eac0804300936j707324afude1c18ce12dba71f@mail.gmail.com>
References: <978d1eac0804300936j707324afude1c18ce12dba71f@mail.gmail.com>
Message-ID: <ca471dc20804300951h654fa4can6f91f1e1af21338c@mail.gmail.com>

On Wed, Apr 30, 2008 at 9:36 AM, Farshid Lashkari <flashk at gmail.com> wrote:
>  I was just curious as to why cStringIO objects don't implement the
>  buffer interface.  cStringIO objects seem similar to string and array
>  objects, and those support the buffer protocol. Is there a reason
>  against allowing cStringIO to support at least the read buffer
>  interface, or is just that nobody has considered it until now?

Well, for one, it would mean you could no longer exchange a StringIO
instance for a cStringIO instance.

Also, what's the compelling use case you're thinking of?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tnelson at onresolve.com  Wed Apr 30 19:40:37 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 30 Apr 2008 10:40:37 -0700
Subject: [Python-Dev] socket.try_reuse_address()
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org> 
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E248257A2@EXMBX04.exchhosting.com>


> > Giampaolo Rodola' wrote:
> > > Maybe it would be better considering Windows CE systems too:
> > >
> > > - if os.name == 'nt'
> > > + if os.name in ('nt', 'ce')
> > >
> > Cygwin? I don't know how Unix-like it is.
>
> Yeah, that's a fair point, it's the behaviour of the
> underlying Winsock API we're targeting, so it would apply to
> Cygwin as well.  (And CE and anything else on Windows.)

....including Jython and IronPython -- which all exhibit the same undesirable behaviour on Windows when SO_REUSEADDR is set against a TCP/IP socket.  Updated patch below.  Assuming there are no objections, I'd like to clean this up and commit over the weekend, once I've updated the various parts of the stdlib currently using SO_REUSEADDR, as well as affected documentation.

Index: socket.py
===================================================================
--- socket.py   (revision 62509)
+++ socket.py   (working copy)
@@ -143,8 +143,18 @@
     'sendall', 'setblocking',
     'settimeout', 'gettimeout', 'shutdown')

+# Attempt to determine if we're running on Windows, irrespective of our Python
+# incarnation.  We need to know this so that we *don't* set the SO_REUSEADDR
+# against TCP/IP sockets in socket.try_reuse_addr().  Note that IronPython is
+# covered below as it sets os.name to 'nt'.
 if os.name == "nt":
     _socketmethods = _socketmethods + ('ioctl',)
+    _is_windows = True
+elif os.name == 'java':
+    from java.lang import System
+    _is_windows = 'windows' in System.getProperty('os.name').lower()
+elif os.name == 'posix' and sys.platform == 'cygwin':
+    _is_windows = True

 if sys.platform == "riscos":
     _socketmethods = _socketmethods + ('sleeptaskw',)
@@ -197,6 +207,13 @@
         Return a new socket object connected to the same system resource."""
         return _socketobject(_sock=self._sock)

+    def try_reuse_address(self):
+        if not (_is_windows and self._sock.type != SOCK_DGRAM):
+            try:
+                self._sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
+            except socket.error:
+                pass
+
     def makefile(self, mode='r', bufsize=-1):
         """makefile([mode[, bufsize]]) -> file object



        Trent.

From tnelson at onresolve.com  Wed Apr 30 19:42:48 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 30 Apr 2008 10:42:48 -0700
Subject: [Python-Dev] socket.try_reuse_address()
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org>  
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com>

>  if os.name == "nt":
>      _socketmethods = _socketmethods + ('ioctl',)
> +    _is_windows = True
> +elif os.name == 'java':
> +    from java.lang import System
> +    _is_windows = 'windows' in System.getProperty('os.name').lower()
> +elif os.name == 'posix' and sys.platform == 'cygwin':
> +    _is_windows = True

Oops, that last line should have been:

elif os.name == 'ce' or (os.name == 'posix' and sys.platform == 'cygwin'):
    _is_windows = True

From brett at python.org  Wed Apr 30 19:49:01 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 30 Apr 2008 10:49:01 -0700
Subject: [Python-Dev] Optimization of Python ASTs: How should we deal
	with constant values?
In-Reply-To: <481846D2.6090509@vector-seven.com>
References: <481846D2.6090509@vector-seven.com>
Message-ID: <bbaeab100804301049y6bd95fe1s34f78d169a633947@mail.gmail.com>

On Wed, Apr 30, 2008 at 3:15 AM, Thomas Lee <tom at vector-seven.com> wrote:
> Hi all,
>
>  I've been working on optimization of the AST, including the porting of the
> old bytecode-level optimizations to the AST level. A few questions have come
> up in the process of doing this, all of which are probably appropriate for
> discussion on this list. The code I'm referring to here can be found in the
> tlee-ast-optimize branch. Most of the relevant code is in Python/optimize.c
> and Python/peephole.c.
>
>  Nearly all of the bytecode-level optimizations have been moved to the AST
> optimizer with a few exceptions. Most of those waiting to be ported are
> stuck in limbo due to the fact we can't yet inject arbitrary PyObject
> constants into the AST. Examples are tuples of constants and the
> optimization of "LOAD_GLOBAL/LOAD_NAME None" as "LOAD_CONST None".
>
>  This leaves us with a few options:
>
>  1. A new AST expr node for constant values for types other than Str/Num
>
>  I imagine this to be something like Const(PyObject* v), which is
> effectively translated to a "LOAD_CONST v" by the compiler. This trades the
> purity of the AST for a little practicality. A "Const" node has no real
> source representation, it would exist solely for the purpose of injecting
> PyObject constants into the AST.
>

Slight issue with this is that people like Jython are standardizing on
our AST, so adding to it does up their burden if they have no use for
it.

>  2. Allow arbitrary annotations to be applied to the AST as compiler hints.
>
>  For example, each AST node might have an optional dict that contains a set
> of annotation values. Then when traversing the AST, the compiler might do
> something along the lines of:
>
>  if (expr->annotations) {
>   PyObject* constvalue = PyDict_GetItemString(expr->annotations,
> "constantvalue");
>   if (constvalue)
>    ADDOP_O(c, LOAD_CONST, constvalue, consts)
>   else
>    VISIT(c, expr, expr)
>  }
>
>  This is a more general solution if we want to keep other compiler hints
> down the track, but unless somebody can think of another use case this is
> probably overkill.
>

Possibly, but the AST stuff is so new who knows if it truly will be
overkill or not down the road. I personally prefer this approach.

>  3. Keep these particular optimizations at the bytecode level.
>
>  It would be great to be able to perform the optimizations at a higher
> level, but this would require no effort at all. This would mean two passes
> over the same code at two different levels.
>

This might end up being the case, but I would rather avoid the
overhead if possible. The question becomes how many other possible
optimizations might come up that would only work reasonably at the
bytecode level.

And you forgot option 4): ditching the optimizations that only work on
the bytecode. I am not advocating this, but it is an option.

-Brett

From draghuram at gmail.com  Wed Apr 30 20:01:35 2008
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Wed, 30 Apr 2008 14:01:35 -0400
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com>
Message-ID: <2c51ecee0804301101i7648951v78960bc539c54cde@mail.gmail.com>

On Wed, Apr 30, 2008 at 1:42 PM, Trent Nelson <tnelson at onresolve.com> wrote:
> >  if os.name == "nt":
>  >      _socketmethods = _socketmethods + ('ioctl',)
>  > +    _is_windows = True
>  > +elif os.name == 'java':
>  > +    from java.lang import System
>  > +    _is_windows = 'windows' in System.getProperty('os.name').lower()

This one will not work.

>>> 'windows' in System.getProperty('os.name').lower()
Traceback (innermost last):
  File "<console>", line 1, in ?
TypeError: string member test needs char left operand
>>>

You may have to do something like
System.getProperty('os.name').lower().find('windows').

From tnelson at onresolve.com  Wed Apr 30 20:07:37 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 30 Apr 2008 11:07:37 -0700
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <2c51ecee0804301101i7648951v78960bc539c54cde@mail.gmail.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com>
	<2c51ecee0804301101i7648951v78960bc539c54cde@mail.gmail.com>
Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E24825805@EXMBX04.exchhosting.com>


> This one will not work.
>
> >>> 'windows' in System.getProperty('os.name').lower()
> Traceback (innermost last):
>   File "<console>", line 1, in ?
> TypeError: string member test needs char left operand
> >>>

Interesting, which version of Jython were you using?

> You may have to do something like
> System.getProperty('os.name').lower().find('windows').

That didn't work for me.  I assume the following works for you:

        System.getProperty('os.name').lower().startswith('windows')


From draghuram at gmail.com  Wed Apr 30 20:22:49 2008
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Wed, 30 Apr 2008 14:22:49 -0400
Subject: [Python-Dev] socket.try_reuse_address()
In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E24825805@EXMBX04.exchhosting.com>
References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com>
	<b5b97b01-bd98-4443-ad86-97edbcac468a@34g2000hsh.googlegroups.com>
	<fv7ecd$8da$1@ger.gmane.org>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com>
	<2c51ecee0804301101i7648951v78960bc539c54cde@mail.gmail.com>
	<87D3F9C72FBF214DB39FA4E3FE618CDC6E24825805@EXMBX04.exchhosting.com>
Message-ID: <2c51ecee0804301122s730bf86fy31f7cfa4883b6e98@mail.gmail.com>

On Wed, Apr 30, 2008 at 2:07 PM, Trent Nelson <tnelson at onresolve.com> wrote:
>
>  > This one will not work.
>  >
>  > >>> 'windows' in System.getProperty('os.name').lower()
>  > Traceback (innermost last):
>  >   File "<console>", line 1, in ?
>  > TypeError: string member test needs char left operand
>  > >>>
>
>  Interesting, which version of Jython were you using?

2.1. Now that you mentioned it, I tested with 2.2 and your code works there.

>  > You may have to do something like
>  > System.getProperty('os.name').lower().find('windows').
>
>  That didn't work for me.  I assume the following works for you:
>
>         System.getProperty('os.name').lower().startswith('windows')

It does.

From exarkun at divmod.com  Wed Apr 30 20:38:04 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 30 Apr 2008 14:38:04 -0400
Subject: [Python-Dev] cStringIO buffer interface
In-Reply-To: <ca471dc20804300951h654fa4can6f91f1e1af21338c@mail.gmail.com>
Message-ID: <20080430183804.6859.262788219.divmod.quotient.57183@ohm>

On Wed, 30 Apr 2008 09:51:25 -0700, Guido van Rossum <guido at python.org> wrote:
>On Wed, Apr 30, 2008 at 9:36 AM, Farshid Lashkari <flashk at gmail.com> wrote:
>>  I was just curious as to why cStringIO objects don't implement the
>>  buffer interface.  cStringIO objects seem similar to string and array
>>  objects, and those support the buffer protocol. Is there a reason
>>  against allowing cStringIO to support at least the read buffer
>>  interface, or is just that nobody has considered it until now?
>
>Well, for one, it would mean you could no longer exchange a StringIO
>instance for a cStringIO instance.

It would probably only mean that there is one further incompatibility
between cStringIO and StringIO - you already can't exchange them in a
number of cases.  They handle unicode differently, they have different
methods, etc.

Maybe making them diverge even further is a step in the wrong direction,
though.

>Also, what's the compelling use case you're thinking of?

I'm not sure what use-case Farshid Lashkari had.  For Twisted, it has
been considered as a way to reduce peak memory usage (by reducing the
need for memory copying, which also speeds things up).  I'm not sure
if anyone has benchmarked this yet, so I don't know if it's a real win
or not.  I think Thomas Herv? has a patch to cStringIO which implements
the feature, though.

For reference, <http://twistedmatrix.com/trac/ticket/3188>.

This isn't high on my priority list, but I thought I'd point out the
potential use-case.

Jean-Paul

From martin at v.loewis.de  Wed Apr 30 22:36:36 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Apr 2008 22:36:36 +0200
Subject: [Python-Dev] Optimization of Python ASTs: How should we deal
 with	constant values?
In-Reply-To: <481846D2.6090509@vector-seven.com>
References: <481846D2.6090509@vector-seven.com>
Message-ID: <4818D854.1060906@v.loewis.de>

> This leaves us with a few options:

5. Reuse/Abuse Num(object) for arbitrary constants.
   AFAICT, this should work out of the box.

> 1. A new AST expr node for constant values for types other than Str/Num
> 
> I imagine this to be something like Const(PyObject* v), which is
> effectively translated to a "LOAD_CONST v" by the compiler. This trades
> the purity of the AST for a little practicality. A "Const" node has no
> real source representation, it would exist solely for the purpose of
> injecting PyObject constants into the AST.

I think this is the way to go. It doesn't violate purity: it is an
*abstract* syntax, meaning that there doesn't need to be a 1:1
relationship to source syntax. However, it is still possible to
reproduce source code from this Const node.

I also don't worry about Jython conflicts. The grammar has a version
number precisely so that you can refer to a specific version if you
need to.

Regards,
Martin

From lists at cheimes.de  Wed Apr 30 23:17:03 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 30 Apr 2008 23:17:03 +0200
Subject: [Python-Dev] Module properties for C modules
Message-ID: <4818E1CF.7030903@cheimes.de>

Hello Python Dev!

As you all know modules don't support properties. However several places
and modules could use properties on module level. For example the
sys.py3k_warnings flag could be implemented with a property. Other flags
in the sys module could benefit from read only properties, too.

How do you like the general idea of properties for builtin modules. That
is modules written in C like the sys module.

Christian

From musiccomposition at gmail.com  Wed Apr 30 23:22:51 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 30 Apr 2008 16:22:51 -0500
Subject: [Python-Dev] Module properties for C modules
In-Reply-To: <4818E1CF.7030903@cheimes.de>
References: <4818E1CF.7030903@cheimes.de>
Message-ID: <1afaf6160804301422g6beff08aya616eebdda6ee874@mail.gmail.com>

On Wed, Apr 30, 2008 at 4:17 PM, Christian Heimes <lists at cheimes.de> wrote:
> Hello Python Dev!
>
>  As you all know modules don't support properties. However several places
>  and modules could use properties on module level. For example the
>  sys.py3k_warnings flag could be implemented with a property. Other flags
>  in the sys module could benefit from read only properties, too.

Big +1. Frankly, the get/set methods of sys are quite ugly!
>
>  How do you like the general idea of properties for builtin modules. That
>  is modules written in C like the sys module.

Good idea. Perhaps eventually they could be extended to Python, but
they are definitely useful in C now. How about passing a list of
getsets to PyImport_InitModule(5)?

>
>  Christian
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson

From dev.python.heiko at weinen.org  Wed Apr 30 22:55:31 2008
From: dev.python.heiko at weinen.org (Heiko Weinen)
Date: Wed, 30 Apr 2008 22:55:31 +0200
Subject: [Python-Dev] Another GSoC Student Introduction
Message-ID: <20080430205531.GA15767@vortex.local.net>

Hi!

My name is Heiko N. Weinen and I was accepted as GSoC Student, too. Hoorray!
The project i chose is about Filesystem Virtualisation for Python's Standard Library,
hopefully something which will prove quite useful. :)

Christian Heimes is my Mentor and I'd like to thank him right now for his help.

Feel free to contact me, if you have any further questions or related ideas!
I will set up a project site at http://heiko.weinen.org/gsoc08 in a few days.

Regards,
Heiko

-- 
Lubarsky's Law of Cybernetic Entomology: prov.
    "There is always one more bug."

From lists at cheimes.de  Wed Apr 30 23:32:57 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 30 Apr 2008 23:32:57 +0200
Subject: [Python-Dev] Module properties for C modules
In-Reply-To: <1afaf6160804301422g6beff08aya616eebdda6ee874@mail.gmail.com>
References: <4818E1CF.7030903@cheimes.de>
	<1afaf6160804301422g6beff08aya616eebdda6ee874@mail.gmail.com>
Message-ID: <4818E589.1070400@cheimes.de>

Benjamin Peterson schrieb:
> Good idea. Perhaps eventually they could be extended to Python, but
> they are definitely useful in C now. How about passing a list of
> getsets to PyImport_InitModule(5)?

Yeah, I've a similar idea with PyImport_InitModule5() and a list of
structs containing name, getter, setter, deleter, docstring. The module
struct has to gain an additional slot which may contain a dict of names
-> propertyobjects.

Christian

From dickinsm at gmail.com  Wed Apr 30 23:06:01 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 30 Apr 2008 17:06:01 -0400
Subject: [Python-Dev] Debian/alpha test_math failures
Message-ID: <5c6f2a5d0804301406o1f20481fwc28ac7f46e211e93@mail.gmail.com>

Hello all,
test_math is currently failing on the Debian/alpha buildbots (trunk and
py3k).
I've been trying, unsuccessfully, to figure out what's going wrong, and I'm
starting to run out of ideas, so I thought I'd ask the list for help to see
if
anyone has any useful suggestions.

Details of the failure:  running the following code:

from math import log
ar = 9.88e-324
x = log(ar)

produces:

ValueError: math domain error

Somehow, it looks like errno is getting set to something nonzero in
math_1_from_whatever in mathmodule.c, but I really can't figure out
how.

I've tried adding '-mieee' to the gcc compile flags, and I've added a bunch
of autoconf tests to verify that log(9.88e-324) succeeds, produces
roughly the right result, and doesn't set errno to anything nonzero.
All the autoconf tests that should pass do.  So I can't find anything
wrong with the libm implementation of log.

test_math is fine on Tru64/alpha.

Does anyone have access to a Linux/alpha machine, and a few minutes
to figure out exactly what's failing?  Or any suggestions about what might
be failing.  I'm open to wild ideas at this point... :-)

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080430/abd62b60/attachment.htm>

From guido at python.org  Wed Apr 30 23:41:42 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Apr 2008 14:41:42 -0700
Subject: [Python-Dev] Module properties for C modules
In-Reply-To: <4818E1CF.7030903@cheimes.de>
References: <4818E1CF.7030903@cheimes.de>
Message-ID: <ca471dc20804301441q41df8262h9fd2109b9f9a473b@mail.gmail.com>

On Wed, Apr 30, 2008 at 2:17 PM, Christian Heimes <lists at cheimes.de> wrote:
>  As you all know modules don't support properties. However several places
>  and modules could use properties on module level. For example the
>  sys.py3k_warnings flag could be implemented with a property. Other flags
>  in the sys module could benefit from read only properties, too.
>
>  How do you like the general idea of properties for builtin modules. That
>  is modules written in C like the sys module.

But wouldn't this mean that those properties would no longer be
available in the module's __dict__?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From armin.ronacher at active-4.com  Wed Apr 30 23:46:22 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Wed, 30 Apr 2008 21:46:22 +0000 (UTC)
Subject: [Python-Dev] Problems with the new super()
Message-ID: <loom.20080430T210749-341@post.gmane.org>

Hi all,

I blogged about that topic today which turned out to be a very bad idea,
so I summarize it for the mailinglist here to hopefully start a discussion
about the topic, which I think is rather important.

In the last weeks something remarkable happened in the Python3 sources:
self kinda became implicit.  Not in function definitions, but in super
calls.  But not only self: also the class passed to `super`.  That's
interesting because it means that the language shifts into a completely
different direction.

`super` was rarely used in the past, mainly because it was weird to use.
In the most common use case the current class and the current instance
where passed to it, and the super typed returned looked up the parent
methods on the MRO for you.  It was useful for multiple inheritance and
mixin classes that don't know their parent but confusing for many.  I can
see that a replacement is a good idea, but I'm not so sure if the current
implementation is the way to go.

The main problem with replacing `super(Foo, self).bar()` with something
like `super.bar()` is obviously that self is explicit and the class (in
that case Foo) can't be determined by the caller.  Furthermore the Python
principle was always against functions doing stack introspection to find
the caller.  There are few examples in the stdlib or builtins that do
some sort of caller introspection.  Those are the special functions
`vars`, `locals`, `gloabal`, `vars` and some functions in the inspect
module.  And all of them do nothing more than getting the current frame
and accessing the dict of locals or globals.  What super in current
Python 3 builds does goes way beyond that.

The implementation of the automatic super currently has two ugly details
that I think violate the Python Zen:  The bytecode generated is differently
if the name "super" is used in the function.  `__class__` is only added as
cell to the code if `super` or `__class__` is referenced.  That and the fact
that `f_localsplus` is completely unavailable from within python makes the
whole process appear magical.

This is way more magical than anything we've had in Python in the past and
just doesn't fit into the language in my opinion.  We do have an explicit
self in methods and methods are more or less just functions.  Python's
methods are functions, just that a descriptor puts a method object around
it to pass the self as first arguments.  That's an incredible cool thing
to have and makes things very simple and non-magical.  Breaking that
principle by introducing an automatic super seems to harm the concept.

Another odd thing is that Python 3 starts keeping information on the C
layer we can't access from within Python.  Super is one example, another
good one are methods.  They don't have a descriptor that wraps them if
they are accessed via their classes.  This as such is not a problem as you
can call them the same (just that you can call them with completely
different receivers now) but it becomes a problem if some of the functions
are marked as staticmethods.  Then they look completely the same when
looking at them from a classes perspective:

|   >>> class C:
|   ...  normal = lambda x: None
|   ...  static = staticmethod(lambda x: None)
|   ... 
|   >>> type(C.normal) is type(C.static)
|   True
|   >>> C.normal
|   <function <lambda> at 0x4da150>

As far as I can see a documentation tool has no chance to keep them apart
even though they are completely different on an instance:

|   >>> type(C().normal) is type(C().static)
|   False
|   >>> C().normal
|   <bound method C.<lambda> of <__main__.C object at 0x4dbcf0>>
|   >>> C().static
|   <function <lambda> at 0x4da198>

While I don't knwo about the method thing, I think an automatic super should
at least be implementable from within Python.  I could imagine that by adding
__class__ and __self__ to scopes automatically a lot of that magic could be
removed.

Regards,
Armin


From guido at python.org  Wed Apr 30 23:57:32 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Apr 2008 14:57:32 -0700
Subject: [Python-Dev] Problems with the new super()
In-Reply-To: <loom.20080430T210749-341@post.gmane.org>
References: <loom.20080430T210749-341@post.gmane.org>
Message-ID: <ca471dc20804301457m56c41a46w6cb394f9bf34bdd7@mail.gmail.com>

The staticmethod thing isn't new; that's also the case in 2.x.

The super() thing is a case of practicality beats purity. Note that
you pay a small but measurable cost for the implicit __class__ (it's
implemented as a "cell variable", the same mechanism used for nested
scopes) so we wouldn't want to introduce it unless it is used.

On Wed, Apr 30, 2008 at 2:46 PM, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hi all,
>
>  I blogged about that topic today which turned out to be a very bad idea,
>  so I summarize it for the mailinglist here to hopefully start a discussion
>  about the topic, which I think is rather important.
>
>  In the last weeks something remarkable happened in the Python3 sources:
>  self kinda became implicit.  Not in function definitions, but in super
>  calls.  But not only self: also the class passed to `super`.  That's
>  interesting because it means that the language shifts into a completely
>  different direction.
>
>  `super` was rarely used in the past, mainly because it was weird to use.
>  In the most common use case the current class and the current instance
>  where passed to it, and the super typed returned looked up the parent
>  methods on the MRO for you.  It was useful for multiple inheritance and
>  mixin classes that don't know their parent but confusing for many.  I can
>  see that a replacement is a good idea, but I'm not so sure if the current
>  implementation is the way to go.
>
>  The main problem with replacing `super(Foo, self).bar()` with something
>  like `super.bar()` is obviously that self is explicit and the class (in
>  that case Foo) can't be determined by the caller.  Furthermore the Python
>  principle was always against functions doing stack introspection to find
>  the caller.  There are few examples in the stdlib or builtins that do
>  some sort of caller introspection.  Those are the special functions
>  `vars`, `locals`, `gloabal`, `vars` and some functions in the inspect
>  module.  And all of them do nothing more than getting the current frame
>  and accessing the dict of locals or globals.  What super in current
>  Python 3 builds does goes way beyond that.
>
>  The implementation of the automatic super currently has two ugly details
>  that I think violate the Python Zen:  The bytecode generated is differently
>  if the name "super" is used in the function.  `__class__` is only added as
>  cell to the code if `super` or `__class__` is referenced.  That and the fact
>  that `f_localsplus` is completely unavailable from within python makes the
>  whole process appear magical.
>
>  This is way more magical than anything we've had in Python in the past and
>  just doesn't fit into the language in my opinion.  We do have an explicit
>  self in methods and methods are more or less just functions.  Python's
>  methods are functions, just that a descriptor puts a method object around
>  it to pass the self as first arguments.  That's an incredible cool thing
>  to have and makes things very simple and non-magical.  Breaking that
>  principle by introducing an automatic super seems to harm the concept.
>
>  Another odd thing is that Python 3 starts keeping information on the C
>  layer we can't access from within Python.  Super is one example, another
>  good one are methods.  They don't have a descriptor that wraps them if
>  they are accessed via their classes.  This as such is not a problem as you
>  can call them the same (just that you can call them with completely
>  different receivers now) but it becomes a problem if some of the functions
>  are marked as staticmethods.  Then they look completely the same when
>  looking at them from a classes perspective:
>
>  |   >>> class C:
>  |   ...  normal = lambda x: None
>  |   ...  static = staticmethod(lambda x: None)
>  |   ...
>  |   >>> type(C.normal) is type(C.static)
>  |   True
>  |   >>> C.normal
>  |   <function <lambda> at 0x4da150>
>
>  As far as I can see a documentation tool has no chance to keep them apart
>  even though they are completely different on an instance:
>
>  |   >>> type(C().normal) is type(C().static)
>  |   False
>  |   >>> C().normal
>  |   <bound method C.<lambda> of <__main__.C object at 0x4dbcf0>>
>  |   >>> C().static
>  |   <function <lambda> at 0x4da198>
>
>  While I don't knwo about the method thing, I think an automatic super should
>  at least be implementable from within Python.  I could imagine that by adding
>  __class__ and __self__ to scopes automatically a lot of that magic could be
>  removed.
>
>  Regards,
>  Armin
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gvwilson at cs.utoronto.ca  Wed Apr 23 01:08:12 2008
From: gvwilson at cs.utoronto.ca (Greg Wilson)
Date: Tue, 22 Apr 2008 23:08:12 -0000
Subject: [Python-Dev] Encoding detection in the standard library?
In-Reply-To: <08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>
References: <79235B51-32E4-4E88-AE15-CB1A8A621943@cs.toronto.edu>
	<18444.50265.618909.751390@montanaro-dyndns-org.local>
	<91A34FC2-2DAA-45D7-8EEC-91BFF42BDC31@cs.toronto.edu>
	<480D079A.2020904@v.loewis.de>
	<91478133-664C-4F29-90CB-D0985BE01F24@cs.toronto.edu>
	<480D69E9.7020004@v.loewis.de>
	<091D7293-363C-4766-8931-CFE6A9E72331@cs.toronto.edu>
	<480E2918.7090006@v.loewis.de>
	<08Apr22.144234pdt."58696"@synergy1.parc.xerox.com>
Message-ID: <Pine.LNX.4.64.0804221905590.5696@apps3.cs.toronto.edu>

> Bill Janssen:
> Since the site that receives the POST doesn't necessarily have access to 
> the Web page that originally contained the form, that's not really 
> helpful.  However, POSTs can use the MIME type "multipart/form-data" for 
> non-Latin-1 content, and should.  That contains facilities for 
> indicating the encoding and other things as well.

Yup, but DrProject (the target application) also serves as a relay and 
archive for email.  We have no control over the agent used for 
composition, and AFAIK there's no standard way to include encoding 
information.

Thanks,
Greg