From ziade.tarek at gmail.com  Fri Jan  1 16:07:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:07:20 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04D81D51@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<94bdd2610912141458m52da21ear4970506a37214e6@mail.gmail.com>
	<4dab5f760912230700l38a597f7l855bb2304025d6d5@mail.gmail.com>
	<F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
Message-ID: <94bdd2611001010707h4043ff64x7476049e435852b@mail.gmail.com>

2009/12/23 Glyph Lefkowitz <glyph at twistedmatrix.com>:
[..]
> 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages".
>
> The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. ?In other words it's got "bin", "lib", "share", "etc", etc.. ?~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation.

Right, good idea

Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Fri Jan  1 16:32:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:32:27 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
Message-ID: <94bdd2611001010732o38a563bcu6d0cc9122ed5c88f@mail.gmail.com>

On Sat, Dec 12, 2009 at 9:02 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> A while ago I've proposed to refactor the APIs that provides access to
> the installation paths and configuration variables at runtime into a
> single module called "sysconfig", and make it easier for all
> implementations to work with them.

I've applied the suggestions made in this thread.

If no one objects, here's what I am going to do now:

I'll merge this change in the trunk, (I'll drop the branch changesets
in favor of one single, clean changeset) and start to work on the
corresponding doc, so more people will be able to give some feedback
on the APIs once the second 2.7 alpha is out.

Then, in a second step, I'll work on the removal of the Makefile and
pyconfig.h dependencies by adding a _synconfig.py file as suggested
earlier, that will be created during the build process.

Regards,
Tarek


From status at bugs.python.org  Fri Jan  1 18:07:11 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jan 2010 18:07:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100101170711.A5AAF78654@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/25/09 - 01/01/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2537 open (+22) / 16902 closed (+19) / 19439 total (+41)

Open issues with patches:  1016

Average duration of open issues: 705 days.
Median duration of open issues: 462 days.

Open Issues Breakdown
   open  2504 (+22)
pending    32 ( +0)

Issues Created Or Reopened (42)
_______________________________

doc: Code is not always colored                                  12/30/09
CLOSED http://bugs.python.org/issue7487    reopened flox                          
                                                                               

documention buglet for PyBuffer                                  12/26/09
CLOSED http://bugs.python.org/issue7577    created  ronaldoussoren                
       easy                                                                    

Behavior of operations on a closed file object is not documented 12/26/09
       http://bugs.python.org/issue7578    created  blep                          
                                                                               

Patch to add docstrings to msvcrt                                12/26/09
       http://bugs.python.org/issue7579    created  brian.curtin                  
       patch                                                                   

distutils makefile from python.org installer doesn't work on OS  12/27/09
       http://bugs.python.org/issue7580    created  bskaplan                      
                                                                               

incomplete doc of zlib                                           12/27/09
CLOSED http://bugs.python.org/issue7581    created  coolwanglu                    
                                                                               

[patch] diff.py to use iso timestamp                             12/27/09
       http://bugs.python.org/issue7582    created  techtonik                     
       patch                                                                   

doctest should normalize tabs when comparing output              12/27/09
       http://bugs.python.org/issue7583    created  techtonik                     
                                                                               

datetime.rfcformat() for Date and Time on the Internet           12/27/09
       http://bugs.python.org/issue7584    created  techtonik                     
                                                                               

[patch] difflib should separate filename from timestamp with tab 12/27/09
       http://bugs.python.org/issue7585    created  techtonik                     
       patch                                                                   

Typo in collections documentation                                12/28/09
CLOSED http://bugs.python.org/issue7586    created  nneonneo                      
                                                                               

Python 3.1.1 source build issues                                 12/28/09
CLOSED http://bugs.python.org/issue7587    created  sharmaag                      
                                                                               

unittest.TestCase.shortDescription isn't short anymore           12/28/09
       http://bugs.python.org/issue7588    created  exarkun                       
                                                                               

setup.py shouldn't try to build nis module when nis headers aren 12/28/09
CLOSED http://bugs.python.org/issue7589    created  Arfrever                      
       patch                                                                   

'exceptions' module mentioned in documentation                   12/28/09
CLOSED http://bugs.python.org/issue7590    created  July                          
                                                                               

test_get_platform fails on 3.1                                   12/28/09
       http://bugs.python.org/issue7591    created  pitrou                        
       buildbot                                                                

ssl module documentation: SSLSocket.unwrap description shown twi 12/28/09
       http://bugs.python.org/issue7592    created  mnewman                       
                                                                               

Computed-goto patch for RE engine                                12/29/09
       http://bugs.python.org/issue7593    created  akuchling                     
       patch, needs review                                                     

shlex refactoring                                                12/29/09
       http://bugs.python.org/issue7594    created  ferringb                      
       patch, needs review                                                     

Doc typo for select.kevent()                                     12/29/09
CLOSED http://bugs.python.org/issue7595    created  whit537                       
                                                                               

test_logging sometimes fails                                     12/29/09
CLOSED http://bugs.python.org/issue7596    created  pitrou                        
                                                                               

curses.use_env implementation error                              12/30/09
       http://bugs.python.org/issue7597    created  kanru                         
       patch                                                                   

Cannot install, problems with assembly?                          12/30/09
CLOSED http://bugs.python.org/issue7598    created  sh.00                         
                                                                               

(c)ElementTree can produce invalid XML                           12/30/09
       http://bugs.python.org/issue7599    created  jwilk                         
                                                                               

interpreter crashes before start                                 12/30/09
CLOSED http://bugs.python.org/issue7600    created  mhh91                         
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc         12/30/09
CLOSED http://bugs.python.org/issue7601    created  sadhak                        
                                                                               

Doc: make clean and make update do not delete or update Doc/tool 12/30/09
CLOSED http://bugs.python.org/issue7602    created  flox                          
                                                                               

There is no 'seq' type                                           12/30/09
CLOSED http://bugs.python.org/issue7603    created  tom66                         
                                                                               

delattr __slots__ inconsistancy                                  12/30/09
CLOSED http://bugs.python.org/issue7604    created  ferringb                      
                                                                               

test_cmd_line fails with non-ascii path                          12/30/09
       http://bugs.python.org/issue7605    created  pitrou                        
                                                                               

test_xmlrpc fails with non-ascii path                            12/30/09
       http://bugs.python.org/issue7606    created  pitrou                        
                                                                               

stringlib fastsearch could be improved on 64-bit builds          12/30/09
       http://bugs.python.org/issue7607    created  pitrou                        
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.             12/30/09
CLOSED http://bugs.python.org/issue7608    created  alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                   12/31/09
CLOSED http://bugs.python.org/issue7609    created  Arfrever                      
       patch                                                                   

Cannot use both read and readline method in same ZipExtFile obje 12/31/09
       http://bugs.python.org/issue7610    created  lucifer                       
                                                                               

shlex not posix compliant when parsing "foo#bar"                 12/31/09
       http://bugs.python.org/issue7611    created  jjdmol2                       
                                                                               

Fix "pass and object" typo in Library Reference / Built-in Types 12/31/09
CLOSED http://bugs.python.org/issue7612    created  ygale                         
       patch                                                                   

[cppcheck] found a regression : Invalid number of character ((). 12/31/09
CLOSED http://bugs.python.org/issue7613    created  ettl.martin                   
       patch                                                                   

Python 2.6.4 segfaults                                           12/31/09
CLOSED http://bugs.python.org/issue7614    created  ttsiod                        
                                                                               

unicode_escape codec does not escape quotes                      01/01/10
       http://bugs.python.org/issue7615    created  rhansen                       
       patch                                                                   

test_memoryview test_setitem_writable failures with Intel ICC    01/01/10
       http://bugs.python.org/issue7616    created  ivank                         
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option 01/01/10
       http://bugs.python.org/issue7617    created  Arfrever                      
       patch                                                                   



Issues Now Closed (46)
______________________

True division of integers could be more accurate                  715 days
       http://bugs.python.org/issue1811    mark.dickinson                
       patch                                                                   

API to clear most free lists                                      690 days
       http://bugs.python.org/issue2019    amaury.forgeotdarc            
       patch                                                                   

unit test UnicodeWarning                                          687 days
       http://bugs.python.org/issue2100    ezio.melotti                  
                                                                               

test_disutils fails                                               568 days
       http://bugs.python.org/issue3054    ronaldoussoren                
       patch                                                                   

test_formatdate_usegmt fails on non-englisch locale               451 days
       http://bugs.python.org/issue4046    r.david.murray                
                                                                               

"??" in u"foo" raises a misleading error                          410 days
       http://bugs.python.org/issue4328    r.david.murray                
                                                                               

zipfile - add __exit__ attribute to make ZipFile object compatib  287 days
       http://bugs.python.org/issue5511    ezio.melotti                  
       patch                                                                   

test_urllib2 fails - urlopen error file not on local host         271 days
       http://bugs.python.org/issue5625    orsenthil                     
                                                                               

Improve --with-dbmliborder option                                 170 days
       http://bugs.python.org/issue6491    benjamin.peterson             
       patch                                                                   

Move the special-case for integer objects out of PyBytes_FromObj  141 days
       http://bugs.python.org/issue6687    alexandre.vassalotti          
       patch, 26backport                                                       

setup.py fails to find headers of system libffi                   104 days
       http://bugs.python.org/issue6943    benjamin.peterson             
       patch, easy                                                             

The replacement suggested for callable(x) in py3k is not	equival   92 days
       http://bugs.python.org/issue7006    benjamin.peterson             
       patch                                                                   

C/API - Document exceptions                                        89 days
       http://bugs.python.org/issue7033    lekma                         
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi   35 days
       http://bugs.python.org/issue7381    georg.brandl                  
       patch                                                                   

optparse Documentation References Example Files that do not Exis   30 days
       http://bugs.python.org/issue7404    georg.brandl                  
       patch                                                                   

datetime.datetime.isoformat truncation problem                     29 days
       http://bugs.python.org/issue7413    amaury.forgeotdarc            
       patch, needs review                                                     

doc: Code is not always colored                                     0 days
       http://bugs.python.org/issue7487    georg.brandl                  
                                                                               

float('nan')**2 != nan. float('nan')**2 error 33 on windows        13 days
       http://bugs.python.org/issue7534    mark.dickinson                
       patch                                                                   

If a generator raises TypeError when being unpacked, an unrelate   10 days
       http://bugs.python.org/issue7548    amaury.forgeotdarc            
                                                                               

ctypes doc improvement: c_char_p                                    6 days
       http://bugs.python.org/issue7569    georg.brandl                  
                                                                               

Strange issue : cursor.commit() with sqlite                         5 days
       http://bugs.python.org/issue7572    ghaering                      
                                                                               

tes_math fails Mac OS X 10.4 due to OverflowError in test_mtestf    5 days
       http://bugs.python.org/issue7575    mark.dickinson                
       patch                                                                   

documention buglet for PyBuffer                                     2 days
       http://bugs.python.org/issue7577    georg.brandl                  
       easy                                                                    

incomplete doc of zlib                                              0 days
       http://bugs.python.org/issue7581    amaury.forgeotdarc            
                                                                               

Typo in collections documentation                                   0 days
       http://bugs.python.org/issue7586    georg.brandl                  
                                                                               

Python 3.1.1 source build issues                                    0 days
       http://bugs.python.org/issue7587    amaury.forgeotdarc            
                                                                               

setup.py shouldn't try to build nis module when nis headers aren    1 days
       http://bugs.python.org/issue7589    benjamin.peterson             
       patch                                                                   

'exceptions' module mentioned in documentation                      1 days
       http://bugs.python.org/issue7590    georg.brandl                  
                                                                               

Doc typo for select.kevent()                                        0 days
       http://bugs.python.org/issue7595    georg.brandl                  
                                                                               

test_logging sometimes fails                                        1 days
       http://bugs.python.org/issue7596    ezio.melotti                  
                                                                               

Cannot install, problems with assembly?                             0 days
       http://bugs.python.org/issue7598    ezio.melotti                  
                                                                               

interpreter crashes before start                                    0 days
       http://bugs.python.org/issue7600    r.david.murray                
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            1 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

Doc: make clean and make update do not delete or update Doc/tool    0 days
       http://bugs.python.org/issue7602    georg.brandl                  
                                                                               

There is no 'seq' type                                              0 days
       http://bugs.python.org/issue7603    benjamin.peterson             
                                                                               

delattr __slots__ inconsistancy                                     0 days
       http://bugs.python.org/issue7604    benjamin.peterson             
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.                0 days
       http://bugs.python.org/issue7608    alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                      1 days
       http://bugs.python.org/issue7609    benjamin.peterson             
       patch                                                                   

Fix "pass and object" typo in Library Reference / Built-in Types    0 days
       http://bugs.python.org/issue7612    ezio.melotti                  
       patch                                                                   

[cppcheck] found a regression : Invalid number of character (().    0 days
       http://bugs.python.org/issue7613    ezio.melotti                  
       patch                                                                   

Python 2.6.4 segfaults                                              0 days
       http://bugs.python.org/issue7614    mark.dickinson                
                                                                               

Fwd: Debian Bug#42318: urllib.py has problems with malformed pro 3436 days
       http://bugs.python.org/issue210849  shinnosuke                    
                                                                               

urllib / urllib2 should cache 301 redirections                   2425 days
       http://bugs.python.org/issue735515  pitrou                        
                                                                               

fast modular exponentiation                                      2084 days
       http://bugs.python.org/issue936813  mark.dickinson                
       patch                                                                   

bdist_deb - Debian packager                                       316 days
       http://bugs.python.org/issue1054967 astraw                        
       patch                                                                   

Carbon.Scrap.PutScrapFlavor                                       987 days
       http://bugs.python.org/issue1700507 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 11 Add Option to Bind to a Local IP Address in httplib.py           462 days
open    http://bugs.python.org/issue3972   

  8 fast modular exponentiation                                     2084 days
closed  http://bugs.python.org/issue936813 

  6 [patch] difflib should separate filename from timestamp with ta    5 days
open    http://bugs.python.org/issue7585   

  6 [patch] diff.py to use iso timestamp                               5 days
open    http://bugs.python.org/issue7582   

  6 Implement fastsearch algorithm for rfind/rindex                   24 days
open    http://bugs.python.org/issue7462   

  5 test_xmlrpc fails with non-ascii path                              2 days
open    http://bugs.python.org/issue7606   

  5 test_logging sometimes fails                                       1 days
closed  http://bugs.python.org/issue7596   

  5 Patch to add docstrings to msvcrt                                  6 days
open    http://bugs.python.org/issue7579   

  5 Distutils-based installer does not detect 64bit versions of Pyt  126 days
open    http://bugs.python.org/issue6792   

  5 _sha256 et al. encode to UTF-8 by default                         17 days
open    http://bugs.python.org/issue3745   




From stefan_ml at behnel.de  Sun Jan  3 09:11:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Jan 2010 09:11:16 +0100
Subject: [Python-Dev] Providing support files to assist 3.x extension
	authors
In-Reply-To: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
References: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
Message-ID: <hhpjf3$lk1$1@ger.gmane.org>

Case Vanhorsen, 20.12.2009 01:38:
> When I ported gmpy (Python to GMP multiple precision library) to
> Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I
> found the code to slightly faster and cleaner than using PyLong_AsLong
> and checking for overflow.

You might want to look at the code Cython generates for integer type 
conversions. We use specialised coercion code that gets generated 
on-the-fly to convert Python long/int from and to all sorts of C integer 
types with compile time (portability) and runtime size/value checks. 
Depending on your needs, this may or may not be faster than the above C-API 
function.

Stefan



From david.lyon at preisshare.net  Mon Jan  4 00:42:15 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 10:42:15 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
Message-ID: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>


> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>
> This new operator removes the ambiguity the original proposal had,
> without making it more
> complex for common use cases. So if you dislike it, you will need to
> propose something
> else that also fixes the ambiguity we had.

Ok.

> Environment markers
>..
>Here are some example of fields using such markers:
>
>Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'

  Requires-Dist: [Windows] pywin32 1.0+

That's simpler, shorter, and less ambiguous. Easier to
parse for package managers.

David





From python at mrabarnett.plus.com  Mon Jan  4 01:14:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 04 Jan 2010 00:14:43 +0000
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4132F3.7020905@mrabarnett.plus.com>

David Lyon wrote:
>> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>>
>> This new operator removes the ambiguity the original proposal had,
>> without making it more
>> complex for common use cases. So if you dislike it, you will need to
>> propose something
>> else that also fixes the ambiguity we had.
> 
> Ok.
> 
>> Environment markers
>> ..
>> Here are some example of fields using such markers:
>>
>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.
> 
'win32' is more specific than 'Windows' and, to me, '1.0+' means
'>=1.0', not '>1.0'.


From martin at v.loewis.de  Mon Jan  4 01:21:53 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 04 Jan 2010 01:21:53 +0100
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4134A1.5090203@v.loewis.de>

>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.

Don't you want the PEP to complete? Why this bike-shedding?

I can agree it's shorter. I can't agree that it's simpler,
or less ambiguous.

Regards,
Martin


From ssteinerx at gmail.com  Mon Jan  4 01:29:14 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 3 Jan 2010 19:29:14 -0500
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
	Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <DF433474-9F8E-40BA-B0B1-D3021CAC6317@gmail.com>


On Jan 3, 2010, at 7:21 PM, Martin v. L?wis wrote:

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>> 
>>  Requires-Dist: [Windows] pywin32 1.0+
>> 
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
> 
> Don't you want the PEP to complete? Why this bike-shedding?

Really....

Enough, already!

S



From david.lyon at preisshare.net  Mon Jan  4 01:47:56 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 11:47:56 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <1349.218.214.45.58.1262566076.squirrel@syd-srv02.ezyreg.com>


Hi Martin, Happy New Year,

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>>
>>   Requires-Dist: [Windows] pywin32 1.0+
>>
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
>
> Don't you want the PEP to complete? Why this bike-shedding?

Well, I'm just helping out by pointing out some simpler ways
as Tarek asked me. I was only answering his question. I was out
celebrating so it took longer to reply than normal.

Bike-Shedding ? Me ? which bikeshed? wanting simple?

Anyway, I'm just reading the PEPs and commenting. If there
are some alterations that can be done, lets discuss them.

> I can agree it's shorter.
> ..

Cool.

What I'd really like is a 'Code-Repository:' keyword
so that we can install programs/libraries directly into
a system.

I feel that this would really simplify the packaging
landscape, making it much easier for the scientific
community and others to do python software installs.

This would allow us to perphaps have something even
*more modern* than CPAN.

So yes, getting PEP-345 right is important to me.

Have a nice day.

David





From abpillai at gmail.com  Mon Jan  4 10:08:05 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Mon, 4 Jan 2010 14:38:05 +0530
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <8548c5f31001040108v635a78ech33521371c6c50e82@mail.gmail.com>

On Sun, Dec 27, 2009 at 11:21 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Tue, Dec 15, 2009 at 12:37 AM, Steven Bethard
> <steven.bethard at gmail.com> wrote:
> >
> > If you're only concerned about 2.X, then yes, optparse will *never* be
> > removed from 2.X. There will be a deprecation note in the 2.X
> > documentation but deprecation warnings will only be issued when the -3
> > flag is specified. Please see the "Deprecation of optparse" section of
> > the PEP:
> >
> > http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
>
 Ref http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse .

 Considering that optparse will be deprecated like 5 years from now.
 I think this point is moot. The deprecation strategy IMHO is
 perhaps way too conservative. Maybe it should be deprecated
 faster ;)




-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100104/e0cabd8c/attachment-0001.html>

From fetchinson at googlemail.com  Mon Jan  4 10:32:10 2010
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 4 Jan 2010 10:32:10 +0100
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <fbe2e2101001040132o22302d2ct72b705ec046bcc46@mail.gmail.com>

>> If you're only concerned about 2.X, then yes, optparse will *never* be
>> removed from 2.X. There will be a deprecation note in the 2.X
>> documentation but deprecation warnings will only be issued when the -3
>> flag is specified. Please see the "Deprecation of optparse" section of
>> the PEP:
>>
>> http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
> Compare:
> http://docs.python.org/library/optparse.html
> http://argparse.googlecode.com/svn/tags/r101/doc/index.html
>
> argparse should be recommended as advanced and more featured variant
> of optparse - that goes without saying, but forcing people to switch
> and annoying them with deprecation messages brings only headache.

As has been noted already nobody is forcing people to switch. Optparse
will be available as a separate package and everybody will be free to
install it and will not have any deprecation messages anywhere.

> Just
> like optparse is better getopt, the latter could also be useful for
> people coming from other languages and porting their libraries to
> Python.
>
> I would better concentrate on real code examples how argparse solves
> problems and would really want to see
> http://www.python.org/dev/peps/pep-0389/#out-of-scope-various-feature-requests
> implemented until argparse enters phase where backward incompatible
> changes in API are impossible.
>
> I would prefer to see PEP 389 as a document describing proposed
> solutions to argument parsing problems rather than petition to replace
> one library with another. So, it should display common argument
> parsing scenarios (use cases) with examples that are also useful
> recipes for documentation. I guess more than 90% people here doesn't
> have time to read argparse methods descriptions to see what they could
> be useful for.



-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From olemis at gmail.com  Mon Jan  4 15:24:05 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 4 Jan 2010 09:24:05 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
Message-ID: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>

On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
> Thanks for the quick response.
>
> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> When I made that change, I didn't know that the __unittest "hack" was
>> being used elsewhere outside of unittest, so I felt fine replacing it
>> with another. While I still consider it an implementation detail, I
>> would be ok with exposing an "official" API for this. Perhaps
>> __unittest_ignore_traceback?
>
> Well, bazaar has had the trick for a couple of years, and googling
> around now turns up some other projects using it or thinking about it:
> <http://github.com/gfxmonk/mocktest/commit/b5e94f7ee06ab627cea2c9cf90d0aeb63ee5a698>
> <http://bitbucket.org/uche/amara/changeset/eeaf69f48271/>
> <http://twistedmatrix.com/trac/ticket/4127>
>

Add `dutest` and probably `nose` [2]_ and ...

> Reinstating the old implementation (with the same name) would mean
> that existing code would keep working with Python 2.7

IOW ... if it is removed all the libraries will have to change and
check for the Py version and ... (probably not a big deal depending on
the details of the ?solution?)

> but maybe a
> discussion could start about a new, less hacky, way of doing the same
>

I am strongly -1 for modifying the classes in ?traditional? unittest
module [2]_ (except that I am strongly +1 for the package structure
WITHOUT TOUCHING anything else ...) , and the more I think about it I
am more convinced ... but anyway, this not a big deal (and in the end
what I think is not that relevant either ... :o). So ...

pass

> May not be worthwhile making life more complicated though,
> there aren't *that* many unittest-extending projects.
>

But every library extending `unittest` will do it or otherwise
not-so-useful error messages will be displayed
;o)

PS: Happy New Year ! BTW

.. [1] [feature] nosexml should omit trace frames where `__unittest...
        (http://code.google.com/p/python-nosexml/issues/detail?id=4#c0)

.. [2] Assessment of unittest 2.7 API : new features and opinions
        (http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free new ripit 1.3.3 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/KFwyUTKH0vI/


From juanfhj at gmail.com  Tue Jan  5 17:10:16 2010
From: juanfhj at gmail.com (Juan Fernando Herrera J.)
Date: Tue, 5 Jan 2010 11:10:16 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards compatibility
Message-ID: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>

How about a new python 3 release with (possibly partial) backwards
compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
software hasn't been ported to it. I'm eager to use 3, but paradoxically,
the 3 release makes me rather stuck with 2.6. Excuse me if this has been
suggested in the past.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/7839cf7b/attachment-0003.htm>

From fuzzyman at voidspace.org.uk  Tue Jan  5 17:50:15 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 05 Jan 2010 16:50:15 +0000
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <4B436DC7.8040605@voidspace.org.uk>

On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
> How about a new python 3 release with (possibly partial) backwards 
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way 
> major software hasn't been ported to it. I'm eager to use 3, but 
> paradoxically, the 3 release makes me rather stuck with 2.6. Excuse me 
> if this has been suggested in the past.
>    

I don't know about other developers, but I certainly expected Python 3 
adoption to take a few years due to libraries only gradually making the 
jump. I also expected there to be nervousness and pain around the 
migration as well.

I think in fact that libraries *are* migrating and there are lots of 
encouraging signs. As a community we need to do all we can to promote 
Python 3 adoption, but I don't think there is much reason to be worried 
(or complacent for that matter).

All the best,

Michael Foord

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/d7b6baac/attachment-0003.htm>

From brian.curtin at gmail.com  Tue Jan  5 18:21:31 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jan 2010 11:21:31 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>

On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:

> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
The proper route to take, in my opinion, is to see what 2.x libraries you
are using that are not 3.x compatible, run 2to3 on them, then run their test
suite, and see where you get. Submit a patch or two to the library and see
what happens -- it at least gets the wheels in motion.

I'm sure everyone out there would like to dive into supporting 3.x, but it's
tough to prioritize when you are up to your eyeballs with 2.x bugs in your
tracker. Look at Twisted (
http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3)
-- over 1000 issues, ~5 developers -- 3.x support won't be here tomorrow,
but it's on the horizon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/67a4e6e2/attachment-0003.htm>

From guido at python.org  Tue Jan  5 18:23:08 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jan 2010 09:23:08 -0800
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B436DC7.8040605@voidspace.org.uk>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<4B436DC7.8040605@voidspace.org.uk>
Message-ID: <ca471dc21001050923i2ca37f6ej543faf0981c43b13@mail.gmail.com>

On Tue, Jan 5, 2010 at 8:50 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
>
> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
> I don't know about other developers, but I certainly expected Python 3
> adoption to take a few years due to libraries only gradually making the
> jump. I also expected there to be nervousness and pain around the migration
> as well.
>
> I think in fact that libraries *are* migrating and there are lots of
> encouraging signs. As a community we need to do all we can to promote Python
> 3 adoption, but I don't think there is much reason to be worried (or
> complacent for that matter).
>

Michael is right, but doesn't answer the OP's implied question "why can't
3.x be made backwards compatible with 2.6?"

Unfortunately the answer isn't easy. I wish it was "because we didn't have
time to do it" (since then the solution would be simply to call for more
volunteers to help out) -- but unfortunately, it comes closer to "we have no
idea how to do it."

Py3k was designed to be backwards incompatible, because that gives the most
long-term benefit of the language improvements. (Perhaps a better way of
saying this is that in the design of language improvements,
feature-by-feature backwards compatibility was intentionally not a
requirement, even though keeping most of the language mostly unchanged *was*
a requirement.)

In principle it would be possible to be more backwards compatible at the
syntactic level, but that's not where the pain of porting code lies -- the
major hurdles are typically he new way of thinking about Unicode (bytes vs.
text strings, instead of 8-bit-strings vs. Unicode strings), and the changed
semantics of dictionary keys and related APIs. Since these issues typically
exist across modules (passing strings and dicts between modules is common),
recognizing individual modules as "2.x" vs. "3.x" isn't possible.

Note, by the way, that you won't be "stuck" on 2.6 forever -- 2.7 alpha 1
was already released.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/eaba2f9b/attachment-0003.htm>

From regebro at gmail.com  Tue Jan  5 18:52:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 18:52:20 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <319e029f1001050952o71cb29afh66445550beeb99c2@mail.gmail.com>

On Tue, Jan 5, 2010 at 17:10, Juan Fernando Herrera J.
<juanfhj at gmail.com> wrote:
> I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.

Yes. Python 3 is not what you want to use today if you write
applications. If you write libraries 2010 is the year to port, IMO.
With some luck, 2011 will be the year to start porting applications
properly. We'll see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ianb at colorstudy.com  Tue Jan  5 20:00:49 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 13:00:49 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
Message-ID: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>

On Tue, Jan 5, 2010 at 11:21 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:
>
>> How about a new python 3 release with (possibly partial) backwards
>> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
>> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
>> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
>> suggested in the past.
>>
>>
> The proper route to take, in my opinion, is to see what 2.x libraries you
> are using that are not 3.x compatible, run 2to3 on them, then run their test
> suite, and see where you get. Submit a patch or two to the library and see
> what happens -- it at least gets the wheels in motion.
>

It's not even that easy -- libraries can't apply patches for Python 3
compatibility as they usually break Python 2 compatibility.  Potentially
libraries could apply patches that make a codebase 2to3 ready, but from what
I've seen that's more black magic than straight forward updating, as such
patches have to trick 2to3 producing the output that is desired.

The only workable workflow I've seen people propose for maintaining a single
codebase with compatibility across both 2 and 3 is to use such tricks, with
aliases to suppress some 2to3 updates when they are inappropriate, so that
you can run 2to3 on install and have a single canonical Python 2 source.
 Python 2.7 won't help much (even though it is trying) as the introduction
of non-ambiguous constructions like b"" aren't compatible with previous
versions of Python and so can't be used in many libraries (support at least
back to Python 2.5 is the norm for most libraries, I think).

Also, running 2to3 on installation is kind of annoying, as you get source
that isn't itself the canonical source, so to fix bugs you have to look at
the installed source and trace it back to the bug in the original source.

I suspect a reasonable workflow might be possible with hg and maybe patch
queues, but I don't feel familiar enough with those tools to map that out.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/e43ccd78/attachment-0003.htm>

From glyph at twistedmatrix.com  Tue Jan  5 21:24:45 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 5 Jan 2010 15:24:45 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.

From regebro at gmail.com  Tue Jan  5 21:57:35 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 21:57:35 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <319e029f1001051257k3827c52ahcd115b15d9a8ece7@mail.gmail.com>

On Tue, Jan 5, 2010 at 21:24, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It seems like this is a problem to be addressed, then. ?Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

Some of it is about changing the code so 2to3 doesn't have to do ugly
things. For example, always use iteritems(), so that 2to3 knows that
items() can be used without converting it to a list, etc. Then we do
have the problems with unicode vs string vs bytes, where I can't think
of a clever solution except your no-op shims, which admittedly is
fugly . For me that tends to turn into testing everything until the
tests run on all platforms, which I guess is close to "black magic".
Don't know what to do about that.

python-incompatibility is about documenting all differences, and also
how to make code that runs under both 2.6 and 3.0 without 2to3. But I
guess it should be extended into how to spell something that is clean
in both 2.6 and 3.x.

>> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.

Ian: If you have examples of 2to3 updated that are not appropriate
(except the x.items() -> list(x.items()) they would be appreciated.

> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

In my experience it turned out to be less of a problem than I thought.
That is to some extent because the modules I've ported has had good
test coverage, and I have fixed 99.9% of the bugs by making the tests
pass. Developing with Distributes 2to3 support has then been quite
smooth and in most cases the separation between the 2.x source and the
3.x source hasn't been a problem.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan  5 22:07:23 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 22:07:23 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <4B43AA0B.5030308@v.loewis.de>

> It's not even that easy -- libraries can't apply patches for Python 3
> compatibility as they usually break Python 2 compatibility.  Potentially
> libraries could apply patches that make a codebase 2to3 ready, but from
> what I've seen that's more black magic than straight forward updating,
> as such patches have to trick 2to3 producing the output that is desired.

I wouldn't qualify it in that way. It may be necessary occasionally to
trick 2to3, but that's really a bug in 2to3 which you should report, so
that trickery is then a work-around for a bug - something that you may
have to do with other API, as well.

The "black magic" is really more in the parts that 2to3 doesn't touch
at all (because they are inherently not syntactic); these are the
problem areas Guido refers to. The "black magic" then is to make the
same code work unmodified for both 2.x and 3.x.

> The only workable workflow I've seen people propose for maintaining a
> single codebase with compatibility across both 2 and 3 is to use such
> tricks, with aliases to suppress some 2to3 updates when they are
> inappropriate

I think you misunderstand. All this is necessary, but *not* to suppress
2to3 updates. More typically, it is necessary because 2to3 leaves the
code unmodified either way.

> Also, running 2to3 on installation is kind of annoying, as you get
> source that isn't itself the canonical source, so to fix bugs you have
> to look at the installed source and trace it back to the bug in the
> original source.

That's an issue indeed, but one that I would hope that can be fixed by
improved traceback printing. It would be good if such traceback printing
could make it into 2.7.

Regards,
Martin


From martin at v.loewis.de  Tue Jan  5 22:13:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Jan 2010 22:13:07 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <4B43AB63.3000607@v.loewis.de>

> No-op constructions like 'bytes("")' could help for older versions of
> Python, though.  A very, very small runtime shim could provide
> support for these, if 2to3 could be told about it somehow.

You actually don't *need* 2to3 support for that - bytes("") can work
in either version:

2.x:
def bytes(s):
  return s

3.x:
def bytes(s)
  return s.encode("ascii")

On top of that, you *could* as for bytes("") to be converted to b"" in
3.x - and it is indeed possible to tell 2to3 about that, and you don't
even need to modify 2to3's source to do so: it can be part of your
package.

>> Also, running 2to3 on installation is kind of annoying, as you get
>> source that isn't itself the canonical source, so to fix bugs you
>> have to look at the installed source and trace it back to the bug
>> in the original source.
> 
> Given the way tracebacks are built, i.e. from filenames stored in
> .pycs rather than based on where the code was actually loaded in the
> filesystem, couldn't 2to3 could do .pyc rewriting to point at the
> original source?  Sort of like our own version of the #line
> directive? :)

I think it could, but it would be fairly flaky as the pycs
can get lost, and lose that information after regeneration. Also,
it may be confusing in other scenarios.

I'd rather have it generate separate mapper files, and change the
traceback printing to consider these (as an option).

> Seriously though, I find it hard to believe that this is a big
> problem.  The 3.x source looks pretty similar to the 2.x source, and
> it's good to look at both if you're dealing with a 3.x issue.

That's my experience as well - the only challenge is that you can't
cut-n-paste the source URL into an editor.

Regards,
Martin


From ianb at colorstudy.com  Tue Jan  5 22:39:21 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 15:39:21 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B43AA0B.5030308@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com> 
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com> 
	<4B43AA0B.5030308@v.loewis.de>
Message-ID: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>

On Tue, Jan 5, 2010 at 3:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > It's not even that easy -- libraries can't apply patches for Python 3
> > compatibility as they usually break Python 2 compatibility. ?Potentially
> > libraries could apply patches that make a codebase 2to3 ready, but from
> > what I've seen that's more black magic than straight forward updating,
> > as such patches have to trick 2to3 producing the output that is desired.
>
> I wouldn't qualify it in that way. It may be necessary occasionally to
> trick 2to3, but that's really a bug in 2to3 which you should report, so
> that trickery is then a work-around for a bug - something that you may
> have to do with other API, as well.
>
> The "black magic" is really more in the parts that 2to3 doesn't touch
> at all (because they are inherently not syntactic); these are the
> problem areas Guido refers to. The "black magic" then is to make the
> same code work unmodified for both 2.x and 3.x.

Just to clarify, the black magic I'm referring to is things like:

try:
?? ?unicode_ = unicode
except NameError:
?? ?unicode_ = str

and some other aliases like this that are unambiguous and which 2to3
won't touch (if you write them correctly). ?If the porting guide noted
all these tricks (of which several have been developed, and I'm only
vaguely aware of a few) that would be helpful. ?It's not a lot of
tricks, but the tricks are not obvious and 2to3 gets the translation
wrong pretty often without them. ?For instance, when I say str in
Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
str and unicode to str. ?That *nothing* translates to bytes by default
(AFAIK) means that people must either be living in a bytes-free world
(which sure, lots of code does) or they are using tricks not included
in 2to3 itself.


Also replying to Glyph:
> > Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.
>
> Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in
the filesystem, couldn't 2to3 could do .pyc rewriting to point at the
original source? ?Sort of like our own version of the #line directive?
:)
>
> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
then I don't currently develop any code that is "installed", I only
develop code that is directly from a source code checkout, and where
the checkout is put on the path.  I guess I could have something that
automatically builds the code on every edit, and that's not
infeasible.  It's just not fun.  So long as I have to support Python 2
(which is like forever) then adding Python 3 only makes development
that much more complicated and much less fun, with no concrete
benefits.  Which is a terribly crotchety of me.  Sorry.

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker


From martin at v.loewis.de  Tue Jan  5 23:23:53 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 23:23:53 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
Message-ID: <4B43BBF9.2000302@v.loewis.de>

> Just to clarify, the black magic I'm referring to is things like:
> 
> try:
>     unicode_ = unicode
> except NameError:
>     unicode_ = str
> 
> and some other aliases like this that are unambiguous and which 2to3
> won't touch (if you write them correctly).  If the porting guide noted
> all these tricks (of which several have been developed, and I'm only
> vaguely aware of a few) that would be helpful.  It's not a lot of
> tricks, but the tricks are not obvious and 2to3 gets the translation
> wrong pretty often without them.  For instance, when I say str in
> Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
> str and unicode to str.

No, that's not what's happening. Instead, str is not translated at all
(i.e. it's *not* translated to str - it just isn't touched).

Translating unicode to str is always correct, AFAICT.

I'm not quite sure what the magic above is supposed to achieve: in 2.x,
unicode_ becomes an alias to unicode, in 3.x, 2to3 translates unicode
to str, and then the block becomes

try:
  unicode_ = str
except NameError:
  unicode_ = str

so the NameError is actually never triggered.

Could it be that the magic is proposed for a mode where you *don't*
use 2to3?

> That *nothing* translates to bytes by default
> (AFAIK) means that people must either be living in a bytes-free world
> (which sure, lots of code does) or they are using tricks not included
> in 2to3 itself.

By your above definition of "translates", the function "bytes" is
translated to bytes (i.e. it isn't touched by 2to3).

> Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
> then I don't currently develop any code that is "installed", I only
> develop code that is directly from a source code checkout, and where
> the checkout is put on the path.  I guess I could have something that
> automatically builds the code on every edit, and that's not
> infeasible.  It's just not fun.

Depends on where you draw your fun from. It is indeed fun to me, but
I can see why it may not be fun to you. So you could ask me to do it for
you - or you may try to use what I have already done for you, so you
don't have to do it.

> So long as I have to support Python 2
> (which is like forever) then adding Python 3 only makes development
> that much more complicated and much less fun, with no concrete
> benefits.

I doubt it will be *much* more complicated - only a little.
As for concrete benefits: there may be no benefits at this point,
but in the long run, starting now will have saved you a lot of
pressure in the long run, and stop users from switching away from
your packages because of lack of Python 3 support.

Regards,
Martin


From ziade.tarek at gmail.com  Wed Jan  6 01:08:34 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 01:08:34 +0100
Subject: [Python-Dev] PEP 386 and PEP 345
Message-ID: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>

Hi,

I think we've reached a consensus on those two PEPs.

Although, there's one last point that was forgotten in the discussions
: I've introduced "rc" in the pre-releases markers, so PEP 386 is
compatible with Python's own version scheme.  "rc" comes right after
"c" in the sorting. It's slightly redundant with the "c" marker but I
don't think this really matters as long as consumers know how to order
them (a < b < c < rc). I have also stated that "c" is the preferred
marker for third party projects, from PEP 386 point of view.

Is there anything else I can do to make those two PEPs accepted ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From david.lyon at preisshare.net  Wed Jan  6 01:26:34 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:26:34 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <4B43BBF9.2000302@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
	<4B43BBF9.2000302@v.loewis.de>
Message-ID: <1322.218.214.45.58.1262737594.squirrel@syd-srv02.ezyreg.com>

Hi Martin,

> ... but in the long run, starting now will have saved you a lot of
> pressure in the long run, and stop users from switching away from
> your packages because of lack of Python 3 support.

In a production situation it works the other way around. If there's
an application that requires twisted (or whatever package) then most
people would use the appropriate interpreter to match the library.

Since you guys all did your jobs so well :-) doing so is painless.

Because there is so much "comfort" with the existing situation it
makes it an effort for people to move to a different lounge chair.
Namely python 3.

I'd suggest that moving the package set (pypi) to python 3 could
be kicked along with the help of some automated tools. I don't
know what tools you guys have got.

But I am very sure that if code analysis was provided to package
developers on python 3 (so they don't have to run it themselves),
then it would be like an even bigger tv screen in a bigger lounge
room and it would assist in drawing them over.

David







From david.lyon at preisshare.net  Wed Jan  6 01:36:24 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:36:24 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <1337.218.214.45.58.1262738184.squirrel@syd-srv02.ezyreg.com>


Hi Ian,

> The only workable workflow I've seen people propose for maintaining a
> single
> codebase with compatibility across both 2 and 3 is to use such tricks,
> with
> aliases to suppress some 2to3 updates when they are inappropriate, so that
> you can run 2to3 on install and have a single canonical Python 2 source.
>  Python 2.7 won't help much (even though it is trying) as the introduction
> of non-ambiguous constructions like b"" aren't compatible with previous
> versions of Python and so can't be used in many libraries (support at
> least
> back to Python 2.5 is the norm for most libraries, I think).
>
> Also, running 2to3 on installation is kind of annoying, as you get source
> that isn't itself the canonical source, so to fix bugs you have to look at
> the installed source and trace it back to the bug in the original source.
>
> I suspect a reasonable workflow might be possible with hg and maybe patch
> queues, but I don't feel familiar enough with those tools to map that out.

That's why I am saying that we need a Code-Repository: section in PEP-345
Metadata.

Let package developers keep two branches in SCM. One for 2.x and one for 3.x

Let them backport features from their 3.x development series to their 2.x
code base. In exactly the same way that it is done in so many other
developments today.

Keeping multiple branches of code is a very common thing these days. And
there are tools in the outside world (bzr,svn,mercurial) that allow us to
do it very easily.

Let's have that support in python; it will make life easier.

David










From david.lyon at preisshare.net  Wed Jan  6 03:01:22 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 13:01:22 +1100 (EST)
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <1617.218.214.45.58.1262743282.squirrel@syd-srv02.ezyreg.com>

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?

Tarek,

Given that I helped out so much last year with the PEP in discussing
different options, even if they weren't accepted, I really feel that it is
unfair if my name isn't mentioned. It was a huge time sacrifice on my
part.

For example, even if I only managed to explain the version numbering and
clarify how that worked. It did take me some time to do that.

What I did do however, was spend a lot of time with the multiplatform
"Markers". I still think that two short weeks more of "discussion" could
resolve some issues. That discussion went for 4 months on distutils-ml.

Look, major issues aside, can you make the following concessions on
PEP-345 which I only feel will help it, namely:

 1) Source-Repository: specify a code repository to install from

 2) Streamline Requires-Python: by implementing "Markers" as
    noted by the PEP. A marker being something like
    "Requires-Python(windows): lxml". Otherwise remove the
    word marker from the PEP and just replace with "metacode".
    Markers are what were discussed on distutils-ml. Metacode
    is what is described in the PEP.

 3) Remove the inconsistency and platform ambiguities. Especially
    for windows users. For example, "win32" is extremely confusing
    for windows users right now. As more and more systems now are
    64 bit. Use the platform module, instead of the sys module
    for constants. I'll post to distutils-ml on this.

I am certainly not trying to hold this PEP up, and I apologise on
my part for my late attention. I will post to distutils-ml on these
and i promise to keep my comments unheated and unwitty.

Having said that, PEP-345 is *super-important*. A week or two or three
more discussion and the issues can be resolved.

We all just want to focus on being productive. It would be a great
accomplishment for you to get PEP-345 approved and likewise for
me getting mentioned even in a minor role as helping out on a PEP.

So I'm hoping that you can make a few last minute concessions
meaning that we can all happily go on our way in 2010.

Regards

David













From brett at python.org  Wed Jan  6 06:20:30 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Jan 2010 21:20:30 -0800
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <bbaeab101001052120qe888df8o7eb03a61e6c030c7@mail.gmail.com>

On Tue, Jan 5, 2010 at 16:08, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?
>

As you said, consensus has been reached, so just Guido's BDFL stamp of
approval is all I can think of.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/a76cb75b/attachment-0003.htm>

From chris at simplistix.co.uk  Wed Jan  6 12:19:45 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:19:45 +0000
Subject: [Python-Dev] bug triage
Message-ID: <4B4471D1.9020707@simplistix.co.uk>

Hi All,

Is there a high volume of incoming bugs to the Python tracker?
If so, I'd like to help with triaging. I think I have all the necessary 
access, what I'm missing is the knowledge of how to set myself up to get 
notifications of new bugs...

How do I do that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From fuzzyman at voidspace.org.uk  Wed Jan  6 12:24:37 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 06 Jan 2010 11:24:37 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <4B4472F5.8000709@voidspace.org.uk>

On 06/01/2010 11:19, Chris Withers wrote:
> Hi All,
>
> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the 
> necessary access, what I'm missing is the knowledge of how to set 
> myself up to get notifications of new bugs...
>
> How do I do that?
>

Bug triaging is one of Python's "big needs" and anything you do to help 
on this score would be much appreciated. Particularly reviewing new and 
outstanding issues.

I assumed there would be RSS feeds for bug tracker activity but can't 
easily find these on the tracker. There is a bot that posts activity to 
#python-dev, so there must be some way of getting this information.

All the best,

Michael



> cheers,
>
> Chris
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From solipsis at pitrou.net  Wed Jan  6 12:25:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 11:25:16 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <loom.20100106T122258-896@post.gmane.org>


Hi Chris,

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary 
> access, what I'm missing is the knowledge of how to set myself up to get 
> notifications of new bugs...

Do you really want to get such notifications? There may be a lot of them.
If you want however, you can join #python-dev on IRC (irc.freenode.net) where
there's a bot which posts updates of all bugs on the tracker. There's usually
not a lot of discussion going on so you probably won't feel flooded.

In addition to bug triage, what is needed is reviewing of existing patches, as
well as writing patches for issues which haven't been addressed yet :-)

Regards

Antoine.




From chris at simplistix.co.uk  Wed Jan  6 12:30:40 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:30:40 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B447460.7080100@simplistix.co.uk>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't 
> easily find these on the tracker. There is a bot that posts activity to 
> #python-dev, so there must be some way of getting this information.

Yeah, email-out is what I'm really after... I have it for my own Roundup 
instance so it can't be that hard to do ;-)

Roch?, you guys host the bug tracker, right? Is there email-out set up 
for it?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ncoghlan at gmail.com  Wed Jan  6 12:37:23 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 21:37:23 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B4475F3.5040406@gmail.com>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't
> easily find these on the tracker. There is a bot that posts activity to
> #python-dev, so there must be some way of getting this information.

I'm pretty sure the bugs list is still the primary spooled notification
mechanism:
http://mail.python.org/mailman/listinfo/python-bugs-list

There are also the weekly tracker activity summaries that are posted
here to python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From chris at simplistix.co.uk  Wed Jan  6 12:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:41:28 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <4B4476E8.8050709@simplistix.co.uk>

Nick Coghlan wrote:
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

That's what I was after, thanks!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From facundobatista at gmail.com  Wed Jan  6 13:03:08 2010
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 6 Jan 2010 09:03:08 -0300
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <e04bdf311001060403y3b65c647jf82fdc26266a0efe@mail.gmail.com>

On Wed, Jan 6, 2010 at 8:19 AM, Chris Withers <chris at simplistix.co.uk> wrote:

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary
> access, what I'm missing is the knowledge of how to set myself up to get
> notifications of new bugs...

Not notifications, but maybe a way to have a higher look of them for
easy selection:

  http://www.taniquetil.com.ar/cgi-bin/pytickets.py

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ziade.tarek at gmail.com  Wed Jan  6 13:29:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:29:58 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>

On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 06/01/2010 11:19, Chris Withers wrote:
>>
>> Hi All,
>>
>> Is there a high volume of incoming bugs to the Python tracker?
>> If so, I'd like to help with triaging. I think I have all the necessary
>> access, what I'm missing is the knowledge of how to set myself up to get
>> notifications of new bugs...
>>
>> How do I do that?
>>
>
> Bug triaging is one of Python's "big needs" and anything you do to help on
> this score would be much appreciated. Particularly reviewing new and
> outstanding issues.

Another useful triage I think, is to review the oldest bugs (some of
them are > 5 years)
and remove the ones that are not relevant anymore, or duplicate with
newer entries.

Tarek

-- 
Tarek Ziad? | http://ziade.org


From chris at simplistix.co.uk  Wed Jan  6 13:31:08 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 12:31:08 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B44828C.4070201@simplistix.co.uk>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this 
with someone as a paired task for those 2 days...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Wed Jan  6 13:37:55 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:37:55 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B44828C.4070201@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B44828C.4070201@simplistix.co.uk>
Message-ID: <94bdd2611001060437s2d0242aembc669c3263773fea@mail.gmail.com>

On Wed, Jan 6, 2010 at 1:31 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Another useful triage I think, is to review the oldest bugs (some of
>> them are > 5 years)
>> and remove the ones that are not relevant anymore, or duplicate with
>> newer entries.
>
> I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this with
> someone as a paired task for those 2 days...

I'll be doing Distutils stuff but I can probably help around a bit in that task:
Distutils have quite a few old issues so I can tackle those


From roche at upfrontsystems.co.za  Wed Jan  6 13:32:39 2010
From: roche at upfrontsystems.co.za (=?ISO-8859-1?Q?Roch=E9?= Compaan)
Date: Wed, 06 Jan 2010 14:32:39 +0200
Subject: [Python-Dev] bug triage
In-Reply-To: <4B447460.7080100@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B447460.7080100@simplistix.co.uk>
Message-ID: <1262781159.2836.218.camel@didi>

On Wed, 2010-01-06 at 11:30 +0000, Chris Withers wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't 
> > easily find these on the tracker. There is a bot that posts activity to 
> > #python-dev, so there must be some way of getting this information.
> 
> Yeah, email-out is what I'm really after... I have it for my own Roundup 
> instance so it can't be that hard to do ;-)
> 
> Roch?, you guys host the bug tracker, right? Is there email-out set up 
> for it?

We do, but we don't administer it. There are a few administrators taking
care of it and you should be able to reach them by logging your request
here: http://psf.upfronthosting.co.za/roundup/meta/ or post it to the
Infrastructure mailing list: infrastructure at python.org


-- 
Roch? Compaan
Upfront Systems                   http://www.upfrontsystems.co.za



From ncoghlan at gmail.com  Wed Jan  6 13:57:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 22:57:24 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B4488B4.2070208@gmail.com>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I believe someone (Daniel Diniz, maybe?) did do a pass over those some
time in the  last 12 months, so most of the obviously irrelevant ones
that are that old should already be gone. Not to say it isn't worth
doing another pass, just saying not to get disheartened if there aren't
many that can be readily closed.

There are at least a few still kicking around just because they're
difficult to deal with (there's an ancient one to do with one of the
ways circular imports can fail that I occasionally go back and reread
before moving on to something more tractable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rdmurray at bitdance.com  Wed Jan  6 14:43:17 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 08:43:17 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4476E8.8050709@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
	<4B4476E8.8050709@simplistix.co.uk>
Message-ID: <20100106134317.3EF141FB5A6@kimball.webabinitio.net>

On Wed, 06 Jan 2010 11:41:28 +0000, Chris Withers <chris at simplistix.co.uk> wrote:
> Nick Coghlan wrote:
> > I'm pretty sure the bugs list is still the primary spooled notification
> > mechanism:
> > http://mail.python.org/mailman/listinfo/python-bugs-list
> 
> That's what I was after, thanks!

Just for completeness, there's also new-bugs-announce if you want
just *new* bug notification.  That's more for people who want to
watch for bugs they want to become nosy on, though; if you are
doing triage python-bugs-list is what you want.

Please also read http://www.python.org/dev/workflow/ if you haven't
already.  Thanks for being willing to chip in!

--David


From brian.curtin at gmail.com  Wed Jan  6 15:57:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 08:57:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4488B4.2070208@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
Message-ID: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I believe someone (Daniel Diniz, maybe?) did do a pass over those some
> time in the  last 12 months, so most of the obviously irrelevant ones
> that are that old should already be gone. Not to say it isn't worth
> doing another pass, just saying not to get disheartened if there aren't
> many that can be readily closed.
>
> There are at least a few still kicking around just because they're
> difficult to deal with (there's an ancient one to do with one of the
> ways circular imports can fail that I occasionally go back and reread
> before moving on to something more tractable).
>
> Cheers,
> Nick.
>

On the topic of bugs that can be readily closed (literally), I've recently
come across a number of issues which appear to be sitting in a patch or
review stage, but their patches have been committed and the issue remains
open. What is the best course of action there? I'd just go ahead and close
the issue myself but I don't have tracker privileges.

I'm willing to help out with another Daniel Diniz-esque triage sweep if that
would help.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/05d9ebd7/attachment-0003.htm>

From ssteinerx at gmail.com  Wed Jan  6 16:14:20 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 6 Jan 2010 10:14:20 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <7FCF8B19-3465-4392-80CC-BEAEBD926ECA@gmail.com>


On Jan 6, 2010, at 7:29 AM, Tarek Ziad? wrote:

> On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> On 06/01/2010 11:19, Chris Withers wrote:
>>> 
>>> Hi All,
>>> 
>>> Is there a high volume of incoming bugs to the Python tracker?
>>> If so, I'd like to help with triaging. I think I have all the necessary
>>> access, what I'm missing is the knowledge of how to set myself up to get
>>> notifications of new bugs...
>>> 
>>> How do I do that?
>>> 
>> 
>> Bug triaging is one of Python's "big needs" and anything you do to help on
>> this score would be much appreciated. Particularly reviewing new and
>> outstanding issues.
> 
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I was actually thinking about that the other day when I saw that the average age of bugs on the Python tracker was at some hideously large 3 digit number.  

The 'success' statistic would be to bring that down below, say, 100.

S



From skip at pobox.com  Wed Jan  6 17:38:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 6 Jan 2010 10:38:13 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <19268.48245.616046.874126@montanaro.dyndns.org>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> I'm pretty sure the bugs list is still the primary spooled
    Nick> notification mechanism:
    Nick> http://mail.python.org/mailman/listinfo/python-bugs-list

Actually, there is a new-bugs-announce list:

    http://mail.python.org/mailman/listinfo/new-bugs-announce

Skip


From solipsis at pitrou.net  Wed Jan  6 18:56:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 17:56:49 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <hi2it0$q48$1@ger.gmane.org>

Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> On the topic of bugs that can be readily closed (literally), I've
> recently come across a number of issues which appear to be sitting in a
> patch or review stage, but their patches have been committed and the
> issue remains open. What is the best course of action there?

Post a message on the issue asking for info.




From solipsis at pitrou.net  Wed Jan  6 19:09:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 18:09:39 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<hi2it0$q48$1@ger.gmane.org>
Message-ID: <loom.20100106T190650-337@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> > On the topic of bugs that can be readily closed (literally), I've
> > recently come across a number of issues which appear to be sitting in a
> > patch or review stage, but their patches have been committed and the
> > issue remains open. What is the best course of action there?
> 
> Post a message on the issue asking for info.

Ok, I realize my answer might have been a bit terse :-)
The patch might be waiting to be merged in all development branches, or it may
not totally resolve the issue, or perhaps documentation needs to be updated, or
perhaps it is pending a verdict from the buildbots, etc. You can't deduce that
the issue is completely fixed from the simple fact that something has been
committed.

Regards

Antoine Pitrou.






From brett at python.org  Wed Jan  6 20:03:32 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 11:03:32 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>  I believe someone (Daniel Diniz, maybe?) did do a pass over those some
>> time in the  last 12 months, so most of the obviously irrelevant ones
>> that are that old should already be gone. Not to say it isn't worth
>> doing another pass, just saying not to get disheartened if there aren't
>> many that can be readily closed.
>>
>> There are at least a few still kicking around just because they're
>> difficult to deal with (there's an ancient one to do with one of the
>> ways circular imports can fail that I occasionally go back and reread
>> before moving on to something more tractable).
>>
>> Cheers,
>> Nick.
>>
>
> On the topic of bugs that can be readily closed (literally), I've recently
> come across a number of issues which appear to be sitting in a patch or
> review stage, but their patches have been committed and the issue remains
> open. What is the best course of action there? I'd just go ahead and close
> the issue myself but I don't have tracker privileges.
>
>
If a core developer is willing to step forward and vouch for you to get
tracker privileges then I will give them to you. We are trying to give out
tracker privs w/ less time than required to get commit privileges. So as
long as you have helped out on a few issues in a positive and correct way
that should be enough to get one of the regulars who perform triage to
notice.

-Brett



I'm willing to help out with another Daniel Diniz-esque triage sweep if that
> would help.
>
> Brian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/f24c9595/attachment-0003.htm>

From nad at acm.org  Wed Jan  6 21:41:05 2010
From: nad at acm.org (Ned Deily)
Date: Wed, 06 Jan 2010 12:41:05 -0800
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <nad-19B167.12410506012010@news.gmane.org>

In article <4B4475F3.5040406 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't
> > easily find these on the tracker. There is a bot that posts activity to
> > #python-dev, so there must be some way of getting this information.
> 
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

Also, that mailing list (along with most python development related 
mailing lists) is mirrored at gmane.org which means it can also be 
obtained via a newsreader (NNTP) or various RSS feeds.

http://dir.gmane.org/gmane.comp.python.bugs

-- 
 Ned Deily,
 nad at acm.org



From nick.bastin at gmail.com  Wed Jan  6 22:14:54 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:14:54 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
Message-ID: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>

(This may occur on more platforms - I can test on more unix platforms
if the consensus is this is an actual problem and I'm not just a nut)

On freebsd5, if you do a simple ./configure --enable-shared in current
(2.7) trunk, your python shared library will build properly, but all
modules will fail to find the shared library and thus fail to build:

gcc -shared build/temp.freebsd-5.3-RELEASE-i386-2.7/u1/Python/Python-2.7a1/Modules/_struct.o
   -L/u1/tmp/python2.7a1/lib -L/usr/local/lib -lpython2.7 -o
build/lib.freebsd-5.3-RELEASE-i386-2.7/_struct.so
/usr/bin/ld: cannot find -lpython2.7
building '_ctypes_test' extension
...

This of course is because libpython2.7.so is in the current directory
and not (yet) installed in /usr/local/lib.  I've made a very simple
fix for this problem that works, but at least to me smells a bit
funny, which is to modify setup.py to add the following to
detect_modules():

        # If we did --enable-shared, we need to be able to find the library
        # we just built in order to build the modules.
        if platform == 'freebsd5':
            add_dir_to_list(self.compiler_obj.library_dirs, '.')


Which brings me to a few questions:

a) Does this seem like a real problem, or am I missing something obvious?

b) Does this fix seem like the sensible thing to do?  (it seems at
least that we ought to check that the user configured --enable-shared
and only set -L. in that case, if that's possible)

Setting --enable-shared when you actually have a libpython2.7.so in
/usr/local/lib (or whatever --prefix you've selected) is possibly even
more dangerous, because it may succeed in linking against a
differently-built library than what you intended.

--
Nick


From nick.bastin at gmail.com  Wed Jan  6 22:39:17 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:39:17 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <66d0a6e11001061339t38202b70mca1091e1926ecf4b@mail.gmail.com>

On Wed, Jan 6, 2010 at 16:14, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> This of course is because libpython2.7.so is in the current directory
> and not (yet) installed in /usr/local/lib.

One minor correction - as you could see from the compile line, the
actual --prefix in this case is /u1/tmp/python2.7a1, but the libraries
obviously aren't installed there yet either.  Perhaps a better fix
than setting -L. would be to put the shared library in
build/lib.freebsd-5.3-RELEASE-i386-2.7 and add that to the library
path for the linker (the build creates this directory, but installs
nothing in it).

--
Nick


From martin at v.loewis.de  Wed Jan  6 23:21:44 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Jan 2010 23:21:44 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <4B450CF8.8090009@v.loewis.de>

> b) Does this fix seem like the sensible thing to do?

No. Linking in setup.py should use the same options as if the module
was built as *shared* through Modules/Setup, which, IIUC, should use
BLDLIBRARY.

Regards,
Martin


From nick.bastin at gmail.com  Thu Jan  7 01:08:13 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 19:08:13 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B450CF8.8090009@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
Message-ID: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:21, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> b) Does this fix seem like the sensible thing to do?
>
> No. Linking in setup.py should use the same options as if the module
> was built as *shared* through Modules/Setup, which, IIUC, should use
> BLDLIBRARY.

Thanks for that pointer, that makes much more sense.  Indeed,
BLDLIBRARY on FreeBSD* is set to '-L. -lpython$(VERSION)' if you set
--enable-shared, but somehow that piece of information doesn't
propagate into the module build.  More investigation to be done...

--
Nick


From rdmurray at bitdance.com  Thu Jan  7 02:22:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 20:22:42 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
Message-ID: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>


On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On the topic of bugs that can be readily closed (literally), I've recently
> > come across a number of issues which appear to be sitting in a patch or
> > review stage, but their patches have been committed and the issue remains
> > open. What is the best course of action there? I'd just go ahead and close
> > the issue myself but I don't have tracker privileges.
> >
> >
> If a core developer is willing to step forward and vouch for you to get
> tracker privileges then I will give them to you. We are trying to give out
> tracker privs w/ less time than required to get commit privileges. So as
> long as you have helped out on a few issues in a positive and correct way
> that should be enough to get one of the regulars who perform triage to
> notice.
> 
> -Brett

I've done a quick scan of issues Brian is nosy on to refresh my
memory, and I'd say he's definitely been making positive contributions.
I'm willing to volunteer to keep an eye on his triage work for a while
if you grant him tracker privs.

Brian, I assume you'll be cognizant of Antoine's advice about making
sure a bug really should be closed before closing it :)  Hanging out in
#python-dev on freenode while working on issues can be helpful, as well,
since you can quickly ask whoever is there for second opinions on
particular bugs.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Thu Jan  7 02:28:26 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 17:28:26 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com> 
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com> 
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
Message-ID: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com> wrote:

>
> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On the topic of bugs that can be readily closed (literally), I've
> recently
> > > come across a number of issues which appear to be sitting in a patch or
> > > review stage, but their patches have been committed and the issue
> remains
> > > open. What is the best course of action there? I'd just go ahead and
> close
> > > the issue myself but I don't have tracker privileges.
> > >
> > >
> > If a core developer is willing to step forward and vouch for you to get
> > tracker privileges then I will give them to you. We are trying to give
> out
> > tracker privs w/ less time than required to get commit privileges. So as
> > long as you have helped out on a few issues in a positive and correct way
> > that should be enough to get one of the regulars who perform triage to
> > notice.
> >
> > -Brett
>
> I've done a quick scan of issues Brian is nosy on to refresh my
> memory, and I'd say he's definitely been making positive contributions.
> I'm willing to volunteer to keep an eye on his triage work for a while
> if you grant him tracker privs.
>
>
Done for the username brian.curtin (email doesn't match the one Brian
emailed from so do let me know, Brian if this is the right username).
Welcome aboard!

-Brett


> Brian, I assume you'll be cognizant of Antoine's advice about making
> sure a bug really should be closed before closing it :)  Hanging out in
> #python-dev on freenode while working on issues can be helpful, as well,
> since you can quickly ask whoever is there for second opinions on
> particular bugs.
>
> --
> R. David Murray                                      www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/726f5ca0/attachment-0003.htm>

From brian.curtin at gmail.com  Thu Jan  7 02:59:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 19:59:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
	<bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
Message-ID: <cf9f31f21001061759m4fee1cc7g2d9fe8bbfd3cb38e@mail.gmail.com>

On Wed, Jan 6, 2010 at 19:28, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com>wrote:
>
>>
>> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
>> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
>> wrote:
>> > > On the topic of bugs that can be readily closed (literally), I've
>> recently
>> > > come across a number of issues which appear to be sitting in a patch
>> or
>> > > review stage, but their patches have been committed and the issue
>> remains
>> > > open. What is the best course of action there? I'd just go ahead and
>> close
>> > > the issue myself but I don't have tracker privileges.
>> > >
>> > >
>> > If a core developer is willing to step forward and vouch for you to get
>> > tracker privileges then I will give them to you. We are trying to give
>> out
>> > tracker privs w/ less time than required to get commit privileges. So as
>> > long as you have helped out on a few issues in a positive and correct
>> way
>> > that should be enough to get one of the regulars who perform triage to
>> > notice.
>> >
>> > -Brett
>>
>> I've done a quick scan of issues Brian is nosy on to refresh my
>> memory, and I'd say he's definitely been making positive contributions.
>> I'm willing to volunteer to keep an eye on his triage work for a while
>> if you grant him tracker privs.
>>
>>
> Done for the username brian.curtin (email doesn't match the one Brian
> emailed from so do let me know, Brian if this is the right username).
> Welcome aboard!
>
>
Yep, that's the one. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/62a95fa0/attachment-0003.htm>

From python at mrabarnett.plus.com  Thu Jan  7 04:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 07 Jan 2010 03:07:56 +0000
Subject: [Python-Dev] GIL required for _all_ Python calls?
Message-ID: <4B45500C.8090905@mrabarnett.plus.com>

Hi,

I've been wondering whether it's possible to release the GIL in the
regex engine during matching.

I know that it needs to have the GIL during memory-management calls, but
does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
an easy way to find out? Or is it just a case of checking the source
files for mentions of the GIL? The header file for PyList_New, for
example, doesn't mention it!

Thanks


From john.arbash.meinel at gmail.com  Thu Jan  7 04:25:48 2010
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 06 Jan 2010 21:25:48 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B45543C.2090200@gmail.com>

MRAB wrote:
> Hi,
> 
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out? Or is it just a case of checking the source
> files for mentions of the GIL? The header file for PyList_New, for
> example, doesn't mention it!
> 
> Thanks

Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
get concurrent updating of the value, and then the final value is wrong.
(two threads do 5+1 getting 6, rather than 7, and when the decref, you
end up at 4 rather than back at 5).

AFAIK, the only things that don't require the GIL are macro functions,
like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
example, will be increfing and setting the exception state, so certainly
needs the GIL to be held.

John
=:->



From benjamin at python.org  Thu Jan  7 04:32:17 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jan 2010 21:32:17 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45543C.2090200@gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com> <4B45543C.2090200@gmail.com>
Message-ID: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>

2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
> get concurrent updating of the value, and then the final value is wrong.
> (two threads do 5+1 getting 6, rather than 7, and when the decref, you
> end up at 4 rather than back at 5).

Correct.

>
> AFAIK, the only things that don't require the GIL are macro functions,
> like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> example, will be increfing and setting the exception state, so certainly
> needs the GIL to be held.

As a general rule, I would say, no Py* macros are safe without the gil
either (the exception being Py_END_ALLOW_THREADS), since they mutate
Python objects which must be protected.



-- 
Regards,
Benjamin


From guido at python.org  Thu Jan  7 05:29:03 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Jan 2010 20:29:03 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B45543C.2090200@gmail.com> 
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
Message-ID: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>

On Wed, Jan 6, 2010 at 7:32 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> > AFAIK, the only things that don't require the GIL are macro functions,
> > like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> > example, will be increfing and setting the exception state, so certainly
> > needs the GIL to be held.
>
> As a general rule, I would say, no Py* macros are safe without the gil
> either (the exception being Py_END_ALLOW_THREADS), since they mutate
> Python objects which must be protected.

That's keeping it on the safe side, since there are some macros like
PyString_AS_STRING() that are also safe, *if* you are owning at least
one reference to the string object.

At the same time, "no Py* macros" is not quite strong enough, since if
you called PyString_AS_STRING() before releasing the GIL but you don't
own a reference to the string object, the string might be deallocated
behind your back by another thread.

A better rule would be "you may access the memory buffer in a PyString
or PyUnicode object with the GIL released as long as you own a
reference to the string object." Everything else is out of bounds (or
not worth the bother).

--
--Guido van Rossum (python.org/~guido)


From yoann.padioleau at facebook.com  Thu Jan  7 08:24:42 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Wed, 6 Jan 2010 23:24:42 -0800
Subject: [Python-Dev] relation between Python.asdl and Tools/compiler/ast.txt
Message-ID: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>

Hi,

I would like to use astgen.py to generate python classes corresponding to the 
AST of something I have defined in a .asdl file, along the line of what is
apparently done for the python AST itself. I thought astgen.py would
take as an argument a .asdl file, but apparently it instead process a file
called ast.txt. Where does this file come from ? Is it generated from
Python.asdl ?



From johan.gill at agama.tv  Thu Jan  7 10:46:52 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 10:46:52 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
Message-ID: <4B45AD8C.5000405@agama.tv>

Hi devs,
the company where I work has done some work on Python, and the question 
is how this work, owned by the company, can be contributed to the 
community properly. Are there any license issues or other pitfalls we 
need to think about? I imagine that other companies have contributed 
before, so this is probably an already solved problem.

Regards
Johan Gill
Agama Technologies



From regebro at gmail.com  Thu Jan  7 12:12:17 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 12:12:17 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45AD8C.5000405@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
Message-ID: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
> Hi devs,
> the company where I work has done some work on Python, and the question is
> how this work, owned by the company, can be contributed to the community
> properly. Are there any license issues or other pitfalls we need to think
> about? I imagine that other companies have contributed before, so this is
> probably an already solved problem.

I'm not a license lawyer, but typically your company needs to give the
code to the community. Yes, it means it stops owning it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From hodgestar+pythondev at gmail.com  Thu Jan  7 12:28:01 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 7 Jan 2010 13:28:01 +0200
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <fb73205e1001070328j44ac747fu7232a89b559ad0d9@mail.gmail.com>

On Thu, Jan 7, 2010 at 1:12 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

This is incorrect.

The correct information is at http://www.python.org/psf/contrib/.

Schiavo
Simon


From swamy.sangamesh at gmail.com  Thu Jan  7 11:46:59 2010
From: swamy.sangamesh at gmail.com (swamy sangamesh)
Date: Thu, 7 Jan 2010 16:16:59 +0530
Subject: [Python-Dev] test_ctypes failure on AIX 5.3 using python-2.6.2 and
	libffi-3.0.9
Message-ID: <c3369a531001070246s441e159cg35b4cab4e3f65541@mail.gmail.com>

Hi All,

I built the python-2.6.2 with the latest libffi-3.0.9 in AIX 5.3 using xlc
compiler.
When i try to run the ctypes test cases, two failures are seen in
test_bitfields.

*test_ints (ctypes.test.test_bitfields.C_Test) ... FAIL
test_shorts (ctypes.test.test_bitfields.C_Test) ... FAIL*

I have attached the full test case result.

If i change the type c_int and c_short to c_unit and c_ushort of class
"BITS(Structure)" in file
test_bitfields.py then no failures are seen.

Has anyone faced the similar issue or any help is appreciated.


Thanks,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0003.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctype-testcases
Type: application/octet-stream
Size: 22996 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0003.obj>

From ncoghlan at gmail.com  Thu Jan  7 13:23:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 07 Jan 2010 22:23:11 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <4B45D22F.40404@gmail.com>

Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
>> Hi devs,
>> the company where I work has done some work on Python, and the question is
>> how this work, owned by the company, can be contributed to the community
>> properly. Are there any license issues or other pitfalls we need to think
>> about? I imagine that other companies have contributed before, so this is
>> probably an already solved problem.
> 
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

As Simon pointed out, while some organisations do work that way, the PSF
isn't one of them.

The PSF only requires that the code be contributed under a license that
then allows us to turn around and redistribute it under a different open
source license without requesting additional permission from the
copyright holder. For corporate contributions, I believe the contributor
agreement needs to be signed by an authorised agent of the company - the
place to check that would probably be psf at python.org (that's the email
address for the PSF board).

Assuming the subject line relates to the code that you would like to
contribute though, that particular change is unlikely to happen - 2.6 is
in maintenance mode and changing RLock from a Python implementation to
the faster C one is solidly in new feature territory. Although a
backport of the 3.2 C RLock implementation to 2.7 could be useful, I
doubt that backporting code provided by an existing committer would be
the subject of this query :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Thu Jan  7 14:11:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:11:27 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>

On Thu, Jan 7, 2010 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder.

Even if the contributed code as in this case is a method in an
existing file? How does that work, how do they keep ownership of one
method in the threading.py method? :-)

> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)

Ah. I probably misunderstood what the suggested contribution was.
Maybe it was a separate file, which I didn't get.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Thu Jan  7 14:15:09 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 07 Jan 2010 13:15:09 +0000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
Message-ID: <4B45DE5D.3030104@voidspace.org.uk>

On 07/01/2010 13:11, Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 13:23, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>    
>> As Simon pointed out, while some organisations do work that way, the PSF
>> isn't one of them.
>>
>> The PSF only requires that the code be contributed under a license that
>> then allows us to turn around and redistribute it under a different open
>> source license without requesting additional permission from the
>> copyright holder.
>>      
> Even if the contributed code as in this case is a method in an
> existing file? How does that work, how do they keep ownership of one
> method in the threading.py method? :-)
>
>    

When contributing code to Python all work remains copyright the original 
author. The combined work is copyright *everyone*. The PSF has a license 
to distribute it, which is all that is important.

How you meaningfully exercise your ownership over chunks of code is left 
for the reader to determine...

(i.e. copyright and ownership are legal terms that don't necessarily 
mean anything *practical* in these situations.)

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From stefan_ml at behnel.de  Thu Jan  7 14:30:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 14:30:16 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
Message-ID: <hi4nl7$2ej$1@ger.gmane.org>

Guido van Rossum, 07.01.2010 05:29:
> A better rule would be "you may access the memory buffer in a PyString
> or PyUnicode object with the GIL released as long as you own a
> reference to the string object." Everything else is out of bounds (or
> not worth the bother).

Is that a "yes" regarding the OP's original question about releasing the 
GIL during regexp searches?

Stefan



From regebro at gmail.com  Thu Jan  7 14:36:42 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:36:42 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DE5D.3030104@voidspace.org.uk>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
	<4B45DE5D.3030104@voidspace.org.uk>
Message-ID: <319e029f1001070536t49f76899j111212b02c14f192@mail.gmail.com>

On Thu, Jan 7, 2010 at 14:15, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> (i.e. copyright and ownership are legal terms that don't necessarily mean
> anything *practical* in these situations.)

OK, fair enough. :-)
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Thu Jan  7 15:05:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 15:05:23 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <hi4pn2$9so$1@ger.gmane.org>

MRAB, 07.01.2010 04:07:
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER

Py_UNICODE_TOLOWER looks safe to me at first glance.


> or PyErr_SetString?

Certainly not safe.


> Is there an easy way to find out?

Release it and fix any crashes? Note that this isn't a safe solution, 
though, as some GIL requiring code may be platform specific. So a better 
approach might be to extract any obviously problematic stuff from the 
existing code (such as any exception handling, explicit ref-counting or 
object creation), and *then* try to release the GIL.

Stefan



From solipsis at pitrou.net  Thu Jan  7 16:38:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 15:38:31 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <loom.20100107T163459-842@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out?

There is no "easy way" to do so. The only safe way is to examine all the
functions or macros you want to call with the GIL released, and assess whether
it is safe to call them. As already pointed out, no reference count should be
changed, and generally no mutable container should be accessed, except if that
container is known not to be referenced anywhere else (that would be the case
for e.g. a list that your function has created and is busy populating).

I agree that releasing the GIL when doing non-trivial regex searches is a
worthwhile research, so please don't give up immediately :-)

Regards

Antoine Pitrou.




From olemis at gmail.com  Thu Jan  7 19:24:59 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 7 Jan 2010 13:24:59 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
	<24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
Message-ID: <24ea26601001071024m2abd988fuc77f94845d57483a@mail.gmail.com>

On Mon, Jan 4, 2010 at 9:24 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
>> Thanks for the quick response.
>>
>> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> but maybe a
>> discussion could start about a new, less hacky, way of doing the same
>>
>
> I am strongly -1 for modifying the classes in ?traditional? unittest
> module [2]_ (except that I am strongly +1 for the package structure
> WITHOUT TOUCHING anything else ...) , and the more I think about it I
> am more convinced ... but anyway, this not a big deal (and in the end
> what I think is not that relevant either ... :o). So ...
>

IOW, if I had all the freedom to implement it, after the pkg structure
I'd do something like :

{{{
#!python

class TestResult:
    # Everything just the same
    def _is_relevant_tb_level(self, tb):
        return '__unittest' in tb.tb_frame.f_globals

class BetterTestResult(TestResult):
    # Further code ... maybe ;o)
    #
    def _is_relevant_tb_level(self, tb):
        # This or anything else you might want to do ;o)
        #
        globs = tb.tb_frame.f_globals
        is_relevant =  '__name__' in globs and \
            globs["__name__"].startswith("unittest")
        del globs
        return is_relevant
}}}

that's what inheritance is for ;o) ... but quite probably that's not
gonna happen, just a comment .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Ubuntu sustituye GIMP por F-Spot  -
http://feedproxy.google.com/~r/simelo-es/~3/-g48D6T6Ojs/ubuntu-sustituye-gimp-por-f-spot.html


From martin at v.loewis.de  Thu Jan  7 21:24:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:24:32 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi4nl7$2ej$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
	<hi4nl7$2ej$1@ger.gmane.org>
Message-ID: <4B464300.2020204@v.loewis.de>

>> A better rule would be "you may access the memory buffer in a PyString
>> or PyUnicode object with the GIL released as long as you own a
>> reference to the string object." Everything else is out of bounds (or
>> not worth the bother).
> 
> Is that a "yes" regarding the OP's original question about releasing the
> GIL during regexp searches?

No, because the regex engine may also operate on buffers that start
moving around when you release the GIL.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:27:09 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:27:09 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B46439D.9030604@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

I don't think that's possible. The regex engine can also operate on
objects whose representation may move in memory when you don't hold
the GIL (e.g. buffers that get mutated). Even if they stay in place -
if their contents changes, regex results may be confusing.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:31:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:31:21 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
Message-ID: <4B464499.4020305@v.loewis.de>

> I would like to use astgen.py to generate python classes corresponding to the 
> AST of something I have defined in a .asdl file, along the line of what is
> apparently done for the python AST itself. I thought astgen.py would
> take as an argument a .asdl file, but apparently it instead process a file
> called ast.txt. Where does this file come from ? Is it generated from
> Python.asdl ?

astgen.py is not used to process asdl files; ast.txt lives right next to
astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

HTH,
Martin


From foom at fuhm.net  Thu Jan  7 21:36:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 7 Jan 2010 15:36:37 -0500
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B46439D.9030604@v.loewis.de>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>

On Jan 7, 2010, at 3:27 PM, Martin v. L?wis wrote:

>> I've been wondering whether it's possible to release the GIL in the
>> regex engine during matching.
>
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated). Even if they stay in place -
> if their contents changes, regex results may be confusing.

It seems probably worthwhile to optimize for the common case of using  
the regexp engine on an immutable object of type "str" or "bytes", and  
allow releasing the GIL in *that* case, even if you have to keep it  
for the general case.

James

From martin at v.loewis.de  Thu Jan  7 21:45:42 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:45:42 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
Message-ID: <4B4647F6.2060309@v.loewis.de>

>>> I've been wondering whether it's possible to release the GIL in the
>>> regex engine during matching.
>>
>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated). Even if they stay in place -
>> if their contents changes, regex results may be confusing.
> 
> It seems probably worthwhile to optimize for the common case of using
> the regexp engine on an immutable object of type "str" or "bytes", and
> allow releasing the GIL in *that* case, even if you have to keep it for
> the general case.

Right. This problem was the one that I thought of first.

Thinking about these things is fairly difficult (to me, at least), so
I think I could only tell whether I would consider a patch thread-safe
that released the GIL around matching under selected circumstances -
if I had the patch available. I don't see any obvious reason (assuming
Guido's list of conditions holds - i.e. you are holding references to
everything you access).

Regards,
Martin


From ncoghlan at gmail.com  Thu Jan  7 21:48:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 08 Jan 2010 06:48:05 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DECB.7070907@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com> <4B45DECB.7070907@agama.tv>
Message-ID: <4B464885.40806@gmail.com>

Johan Gill wrote:
> Yes, it is the new RLock implementation.
> If I understood this correctly, we should make a patch against trunk if
> anything should be contributed.

Yep.

> Do you mean that we wouldn't need the paperwork for backporting the
> original patch committed to py3k?

Whether or not a contributor agreement was essential for this particular
contribution would depend on how much new code was needed for the
backport, but the bulk of the copyright on the C RLock code would remain
with Antoine regardless.

However, sorting through the legalities of the contributor agreement
really is the best way to make sure every is squared away nice and
neatly from a legal point of view.

After all, even if I was a lawyer (which I'm not, I'm just a developer
with an interest in licensing issues), I still wouldn't be *your* lawyer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Jan  7 21:43:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 20:43:17 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <loom.20100107T214211-130@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated).

Why is it a problem? If we get a buffer through the new buffer API, the object
should ensure that the representation isn't moved away until the buffer is 
released.

Regards

Antoine.




From martin at v.loewis.de  Thu Jan  7 21:59:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:59:29 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B464B31.7040406@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

Ok, here is another problem: SRE_OP_REPEAT uses PyObject_MALLOC,
which requires the GIL (it then also may call PyErr_NoMemory,
which also requires the GIL).

Regards,
Martin


From johan.gill at agama.tv  Thu Jan  7 14:16:59 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 14:16:59 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <4B45DECB.7070907@agama.tv>

On 01/07/2010 01:23 PM, Nick Coghlan wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder. For corporate contributions, I believe the contributor
> agreement needs to be signed by an authorised agent of the company - the
> place to check that would probably be psf at python.org (that's the email
> address for the PSF board).
>
> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)
>
> Regards,
> Nick.
>
>    
Yes, it is the new RLock implementation.
If I understood this correctly, we should make a patch against trunk if 
anything should be contributed.
Do you mean that we wouldn't need the paperwork for backporting the 
original patch committed to py3k?

Regards
Johan Gill
Agama Technologies



From yoann.padioleau at facebook.com  Thu Jan  7 22:07:55 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Thu, 7 Jan 2010 13:07:55 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464499.4020305@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
Message-ID: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>


On Jan 7, 2010, at 12:31 PM, Martin v. L?wis wrote:

>> I would like to use astgen.py to generate python classes corresponding to the 
>> AST of something I have defined in a .asdl file, along the line of what is
>> apparently done for the python AST itself. I thought astgen.py would
>> take as an argument a .asdl file, but apparently it instead process a file
>> called ast.txt. Where does this file come from ? Is it generated from
>> Python.asdl ?
> 
> astgen.py is not used to process asdl files; ast.txt lives right next to
> astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

Yes, I know that. That's why I asked about the relation between ast.txt and Python.adsl.
If internally the parser uses the .adsl, but expose as a reflection mechanism things
that were generated from ast.txt, then there could be a mismatch. Where does ast.txt comes from ? Shouldn't it be generated itself from Python.adsl ?

So we would have

Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py containing all the UnarySub, Expression, classes that represents a Python AST.



> 
> HTH,
> Martin



From martin at v.loewis.de  Thu Jan  7 22:11:36 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 07 Jan 2010 22:11:36 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <loom.20100107T214211-130@post.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
Message-ID: <4B464E08.5020703@v.loewis.de>

>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated).
> 
> Why is it a problem? If we get a buffer through the new buffer API, the object
> should ensure that the representation isn't moved away until the buffer is 
> released.

In 2.7, we currently get the buffer with bf_getreadbuffer. In 3.x, we have

    /* Release the buffer immediately --- possibly dangerous
       but doing something else would require some re-factoring
    */
    PyBuffer_Release(&view);


Even if we do use the new API, and correctly, it still might be
confusing if the contents of the buffer changes underneath.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 22:16:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 22:16:12 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
Message-ID: <4B464F1C.7020404@v.loewis.de>

>> astgen.py is not used to process asdl files; ast.txt lives right
>> next to astgen.py. Instead, the asdl file is processed by
>> Parser/asdl_c.py.
> 
> Yes, I know that. That's why I asked about the relation between
> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
> expose as a reflection mechanism things that were generated from
> ast.txt, then there could be a mismatch. Where does ast.txt comes
> from ? Shouldn't it be generated itself from Python.adsl ?

What you may not be aware of is that Tools/compiler (and the
compiler package that it builds on) are both unused and unmaintained.

If the package stops working correctly - tough luck.

> So we would have
> 
> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
> containing all the UnarySub, Expression, classes that represents a
> Python AST.

No - what actually happens in Python 3.x is this: both the compiler
package and Tools/compiler are removed.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 01:10:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 01:10:35 +0100
Subject: [Python-Dev] Improve open() to support reading file starting with
	an unicode BOM
Message-ID: <201001080110.35800.victor.stinner@haypocalc.com>

Hi,

Builtin open() function is unable to open an UTF-16/32 file starting with a 
BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
file starting with a BOM, read()/readline() returns also the BOM whereas the 
BOM should be "ignored".

See recent issues related to reading an UTF-8 text file including a BOM: #7185 
(csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with 
the UTF-8-SIG encoding, but it's possible to do better.

I propose to improve open() (TextIOWrapper) by using the BOM to choose the 
right encoding. I think that only files opened in read only mode should 
support this new feature. *Read* the BOM in a *write* only file would cause 
unexpected behaviours.

Since my proposition changes the result TextIOWrapper.read()/readline() for 
files starting with a BOM, we might introduce an option to open() to enable 
the new behaviour. But is it really needed to keep the backward compatibility?

I wrote a proof of concept attached to the issue #7651. My patch only changes 
the behaviour of TextIOWrapper for reading files starting with a BOM. It 
doesn't work yet if a seek() is used before the first read.

-- 
Victor Stinner
http://www.haypocalc.com/


From guido at python.org  Fri Jan  8 01:52:20 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 16:52:20 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>

I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
talk. And for the other two, perhaps it would make more sense to have
a separate encoding-guessing function that takes a binary stream and
returns a text stream wrapping it with the proper encoding?

--Guido

On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
> file starting with a BOM, read()/readline() returns also the BOM whereas the
> BOM should be "ignored".
>
> See recent issues related to reading an UTF-8 text file including a BOM: #7185
> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
> the UTF-8-SIG encoding, but it's possible to do better.
>
> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
> right encoding. I think that only files opened in read only mode should
> support this new feature. *Read* the BOM in a *write* only file would cause
> unexpected behaviours.
>
> Since my proposition changes the result TextIOWrapper.read()/readline() for
> files starting with a BOM, we might introduce an option to open() to enable
> the new behaviour. But is it really needed to keep the backward compatibility?
>
> I wrote a proof of concept attached to the issue #7651. My patch only changes
> the behaviour of TextIOWrapper for reading files starting with a BOM. It
> doesn't work yet if a seek() is used before the first read.
>
> --
> Victor Stinner
> http://www.haypocalc.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Fri Jan  8 03:23:08 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 02:23:08 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <4B46970C.9010306@mrabarnett.plus.com>

Guido van Rossum wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
> 
Alternatively, have a universal UTF-8/16/32 encoding, ie one that 
expects UTF-8,
with or without BOM, or UTF-16/32 with BOM.
> 
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
>> See recent issues related to reading an UTF-8 text file including a BOM: #7185
>> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
>> the UTF-8-SIG encoding, but it's possible to do better.
>>
>> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
>> right encoding. I think that only files opened in read only mode should
>> support this new feature. *Read* the BOM in a *write* only file would cause
>> unexpected behaviours.
>>
>> Since my proposition changes the result TextIOWrapper.read()/readline() for
>> files starting with a BOM, we might introduce an option to open() to enable
>> the new behaviour. But is it really needed to keep the backward compatibility?
>>
>> I wrote a proof of concept attached to the issue #7651. My patch only changes
>> the behaviour of TextIOWrapper for reading files starting with a BOM. It
>> doesn't work yet if a seek() is used before the first read.
>>


From nick.bastin at gmail.com  Fri Jan  8 04:11:03 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Thu, 7 Jan 2010 22:11:03 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
Message-ID: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>

I think this problem probably needs to move over to distutils-sig, as
it doesn't seem to be specific to the way that Python itself uses
distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
linux and solaris and automatically adds '.' to the library_dirs, and
I suspect it just needs to do this on FreeBSD as well (adding bsd to
the list of platforms for which this is performed "solves" the
problem, but I don't pretend to know enough about either distutils or
freebsd to determine if this is the correct solution).

--
Nick


From glyph at twistedmatrix.com  Fri Jan  8 04:34:36 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 7 Jan 2010 22:34:36 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>



On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>> 
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".

> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

It *is* crazy, but unfortunately rather common.  Wikipedia has a good description of the issues: <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as being UTF-8, so it's become a convention to do that.  That's not good enough, so you need to guess the encoding as well to make sure, but if there is a BOM and you can otherwise verify that the file is probably UTF-8 encoded, you should discard it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/1bc40870/attachment-0003.htm>

From tseaver at palladion.com  Fri Jan  8 05:17:28 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 Jan 2010 23:17:28 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <hi6bko$d0h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
> linux and solaris and automatically adds '.' to the library_dirs, and
> I suspect it just needs to do this on FreeBSD as well (adding bsd to
> the list of platforms for which this is performed "solves" the
> problem, but I don't pretend to know enough about either distutils or
> freebsd to determine if this is the correct solution).

I wouldn't say it needed discussion on the SIG:  just create a bug
report, with the tentative patch you have worked out, and get it
assigned to Tarek.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGsdQACgkQ+gerLs4ltQ5BMQCgtV8snMXH/6dDwgdN4sIJljLd
koYAoKq6c0tKsRSrITHcygu4Od9FVzF5
=BJaE
-----END PGP SIGNATURE-----



From guido at python.org  Fri Jan  8 05:21:04 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 20:21:04 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
Message-ID: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>

On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>
>
> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
>
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>
> file starting with a BOM, read()/readline() returns also the BOM whereas the
>
> BOM should be "ignored".
>
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>
> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
> description of the issues:
> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
> being UTF-8, so it's become a convention to do that. ?That's not good
> enough, so you need to guess the encoding as well to make sure, but if there
> is a BOM and you can otherwise verify that the file is probably UTF-8
> encoded, you should discard it.

That doesn't make sense. If the file isn't UTF-8 you can't see the
BOM, because the BOM itself is UTF-8-encoded.

(And yes, I know this happens. Doesn't mean we need to auto-guess by
default; there are lots of issues e.g. what should happen after
seeking to offset 0?)

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Fri Jan  8 07:06:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 08 Jan 2010 15:06:16 +0900
Subject: [Python-Dev] Improve open() to support reading file
	starting	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
 > talk.

That doesn't stop many applications from doing it.  Python should
perhaps<wink,nudge> not produce UTF-8 + BOM without a disclaimer of
indemnification against all resulting damage, signed in blood, from
the user for each instance.

But it should do something sane when reading such files.  I can't
really see any harm in throwing it away, especially since use of
ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
IIRC.






From tseaver at palladion.com  Fri Jan  8 07:12:12 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 01:12:12 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <hi6ibr$qag$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>
>> BOM should be "ignored".
>>
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.
> 
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

The BOM should not be seekeable if the file is opened with the proposed
"guess encoding from BOM" mode:  it isn't properly part of the stream at
all in that case.

A UTF-8 BOM is an absurditiy, but it exists *everywhere* in the wild:
Python would do wll to make it as easy as possible to consume such
files, as well as the non-insane versions (UTF-16 / UTF-32 BOMs).  In
the best of all possible worlds, I would just try opening the file so:

  f = open('/path/to/file', 'r', encoding="DWIFM")

and any BOM present would set the encoding for the remainder of the stream..



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGzLsACgkQ+gerLs4ltQ5+cwCdGfycPdj6+cPfD23vH644SpHL
sI0AoLGD7nfgMEJdJhBr90yjQQHfDgcJ
=js+2
-----END PGP SIGNATURE-----



From glyph at twistedmatrix.com  Fri Jan  8 08:55:27 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 8 Jan 2010 02:55:27 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>


On Jan 7, 2010, at 11:21 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> 
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>> 
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>> 
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.  If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal.  It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I think it's pretty clear that the BOM should still be skipped in that case ...



From martin at v.loewis.de  Fri Jan  8 10:05:17 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:05:17 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <4B46F54D.9090103@v.loewis.de>

>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I think what Glyph meant is this: if a file starts with the UTF-8
signature, assume it's UTF-8. Then validate the assumption against the
rest of the file also, and then process it as UTF-8. If the rest clearly
is not UTF-8, assume that the UTF-8 signature is bogus.

I understood this proposal as a general processing guideline, not
something the io library should do (but, say, a text editor).

FWIW, I'm personally in favor of using the UTF-8 signature. If people
consider them crazy talk, that may be because UTF-8 can't possibly have
a byte order - hence I call it a signature, not the BOM. As a signature,
I don't consider it crazy at all. There is a long tradition of having
magic bytes in files (executable files, Postscript, PDF, ... - see
/etc/magic). Having a magic byte sequence for plain text to denote the
encoding is useful and helps reducing moji-bake. This is the reason it's
used on Windows: notepad would normally assume that text is in the ANSI
code page, and for compatibility, it can't stop doing that. So the UTF-8
signature gives them an exit strategy.

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:06:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:06:34 +0100
Subject: [Python-Dev] Improve open() to support reading file	starting
 with an unicode BOM
In-Reply-To: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B46F59A.5060905@v.loewis.de>

> But it should do something sane when reading such files.  I can't
> really see any harm in throwing it away, especially since use of
> ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
> IIRC.

And indeed it does, when you open the file in the utf-8-sig encoding.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 10:08:30 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 10:08:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46970C.9010306@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<4B46970C.9010306@mrabarnett.plus.com>
Message-ID: <201001081008.30904.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 03:23:08, MRAB a ?crit :
> Guido van Rossum wrote:
> > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> > talk. And for the other two, perhaps it would make more sense to have
> > a separate encoding-guessing function that takes a binary stream and
> > returns a text stream wrapping it with the proper encoding?
> 
> Alternatively, have a universal UTF-8/16/32 encoding, ie one that
> expects UTF-8,
> with or without BOM, or UTF-16/32 with BOM.

Do you mean open(filename, encoding="BOM")? I suppose that "BOM" would be a 
magical value specific to read a text file (open(filename, "r")), not a real 
codec?

Otherwise which encoding should be used for open(filename, "w", 
encoding="BOM")?

-- 
Victor Stinner
http://www.haypocalc.com/


From martin at v.loewis.de  Fri Jan  8 10:10:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:10:23 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with	an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <4B46F67F.60604@v.loewis.de>

> Builtin open() function is unable to open an UTF-16/32 file starting with a 
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
> file starting with a BOM, read()/readline() returns also the BOM whereas the 
> BOM should be "ignored".

It depends. If you use the utf-8-sig encoding, it *will* ignore the
UTF-8 signature.

> Since my proposition changes the result TextIOWrapper.read()/readline() for 
> files starting with a BOM, we might introduce an option to open() to enable 
> the new behaviour. But is it really needed to keep the backward compatibility?

Absolutely. And there is no need to produce a new option, but instead
use the existing options: define an encoding that auto-detects the
encoding from the family of BOMs. Maybe you call it encoding="sniff".

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:11:51 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:11:51 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <4B46F6D7.1050301@v.loewis.de>

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.

I'm fairly skeptical that anybody on distutils SIG is interested in
details of the Python build process...

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 11:27:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:27:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <201001081127.44044.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
(...)
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I wrote a new version of my patch (version 3):

 * don't change the default behaviour: use open(filename, encoding="BOM") to 
check the BOM is there is any
 * fix for seek(0): always ignore the BOM
 * add an unit test: check that the right encoding is detect, but also the the 
BOM is ignored (especially after a seek(0))

BOM encoding doesn't work for writing into a file, so open(filename, "w", 
encoding="BOM") raises a ValueError.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:31:37 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:31:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <201001081131.37492.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 01:52:20, Guido van Rossum a ?crit :
> And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

I choosed to modify open()+TextIOWrapper instead of writing a new function 
because I would like to avoid an extra read operation (syscall) on the file. 
With my implementation, no specific read operation is needed to detect the 
BOM. The BOM is simply checked in the first _read_chunk().

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:40:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:40:28 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Improve_open=28=29_to_support_reading?=
 =?iso-8859-1?q?_file_starting_with=09an_unicode_BOM?=
In-Reply-To: <4B46F67F.60604@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
Message-ID: <201001081140.28124.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
> > Builtin open() function is unable to open an UTF-16/32 file starting with
> > a BOM if the encoding is not specified (raise an unicode error). For an
> > UTF-8 file starting with a BOM, read()/readline() returns also the BOM
> > whereas the BOM should be "ignored".
> 
> It depends. If you use the utf-8-sig encoding, it *will* ignore the
> UTF-8 signature.

Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
remove the BOM after the first read (much harder if you use a module like 
ConfigParser or csv).

> > Since my proposition changes the result TextIOWrapper.read()/readline()
> > for files starting with a BOM, we might introduce an option to open() to
> > enable the new behaviour. But is it really needed to keep the backward
> > compatibility?
> 
> Absolutely. And there is no need to produce a new option, but instead
> use the existing options: define an encoding that auto-detects the
> encoding from the family of BOMs. Maybe you call it encoding="sniff".

Good idea, I choosed open(filename, encoding="BOM").

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Fri Jan  8 15:27:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:27:42 +0000 (UTC)
Subject: [Python-Dev] GIL required for _all_ Python calls?
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de>
Message-ID: <hi7fcu$gs7$1@ger.gmane.org>

Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
> 
> Even if we do use the new API, and correctly, it still might be
> confusing if the contents of the buffer changes underneath.

Well, no more confusing than when you compute a SHA1 hash or zlib-
compress the buffer, is it?

Regards

Antoine




From solipsis at pitrou.net  Fri Jan  8 15:34:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:34:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <loom.20100108T153305-228@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any

Well, I think if we implement this the default behaviour *should* be changed.
It looks a bit senseless to have two different "auto-choose" options, one with
encoding=None and one with encoding="BOM".

Regards

Antoine.




From guido at python.org  Fri Jan  8 16:48:44 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:48:44 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi7fcu$gs7$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de> 
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de> <hi7fcu$gs7$1@ger.gmane.org>
Message-ID: <ca471dc21001080748lbc18bbx4c4ec2d3f4f027e@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
>>
>> Even if we do use the new API, and correctly, it still might be
>> confusing if the contents of the buffer changes underneath.
>
> Well, no more confusing than when you compute a SHA1 hash or zlib-
> compress the buffer, is it?

That depends. Algorithms that make exactly one pass over the buffer
will run fine (maybe producing a meaningless result). But the regex
matcher may scan the buffer repeatedly (for backtracking purposes) and
it would take a considerable analysis to prove that cannot mess up its
internal data structures if the data underneath changes. (I give it a
decent chance that it's fine, but since it was written without ever
considering this possibility I'm not 100% sure.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:52:48 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:52:48 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
Message-ID: <ca471dc21001080752r312dd47fx32486a95336160ec@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:55 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
> I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.

And I'm saying that it is, with as much certainty as we can ever guess
the encoding of a file.

> If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal. ?It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

Sure, a Latin-1-encoded file could start with the same pattern that is
a UTF-8-encoded BOM. But at that point, a UTF-16-encoded file is also
valid Latin-1.

The question was in the context of encoding-guessing; if we're
guessing, a UTF-8-encoded BOM cannot signify anything else but UTF-8.
(Ditto for UTF-16 and UTF-32 BOMs.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:54:15 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:54:15 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100108T153305-228@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
Message-ID: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>
>> I wrote a new version of my patch (version 3):
>>
>> ?* don't change the default behaviour: use open(filename, encoding="BOM") to
>> check the BOM is there is any
>
> Well, I think if we implement this the default behaviour *should* be changed.
> It looks a bit senseless to have two different "auto-choose" options, one with
> encoding=None and one with encoding="BOM".

Well there *are* two different auto options: use the environment
variables (LANG etc.) or inspect the contents of the file. I think it
would be useful to have ways to specify both.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:56:46 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:56:46 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <ca471dc21001080756p5cb9f560x2a4443efa441fa7b@mail.gmail.com>

On Fri, Jan 8, 2010 at 1:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that. ?That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>>
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
>
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.
>
> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
>
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Sure. I said "crazy talk" only to stir up discussion. Which worked. :-)

Also, I don't want Python's default behavior to change -- sniffing the
encoding should be a separate option.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:59:45 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:59:45 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi6ibr$qag$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<hi6ibr$qag$1@ger.gmane.org>
Message-ID: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
> The BOM should not be seekeable if the file is opened with the proposed
> "guess encoding from BOM" mode: ?it isn't properly part of the stream at
> all in that case.

This feels about right to me. There are still questions though:
immediately after opening a file with a BOM, what should .tell()
return? And regardless of that, .seek(0) should put the file in that
same initial state.

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Fri Jan  8 17:03:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:03:07 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <loom.20100108T170053-283@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.




From mal at egenix.com  Fri Jan  8 17:25:22 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 17:25:22 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <4B475C72.1010207@egenix.com>

Guido van Rossum wrote:
> On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>>
>>> I wrote a new version of my patch (version 3):
>>>
>>>  * don't change the default behaviour: use open(filename, encoding="BOM") to
>>> check the BOM is there is any
>>
>> Well, I think if we implement this the default behaviour *should* be changed.
>> It looks a bit senseless to have two different "auto-choose" options, one with
>> encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Shouldn't this encoding guessing be a separate function that you call
on either a file or a seekable stream ?

After all, detecting encodings is just as useful to have for non-file
streams. You'd then avoid having to stuff everything into
a single function call and also open up the door for more complex
application specific guess work or defaults.

The whole process would then have two steps:

 1. guess encoding

  import codecs
  encoding = codecs.guess_file_encoding(filename)

 2. open the file with the found encoding

  f = open(filename, encoding=encoding)

For seekable streams f, you'd have:

 1. guess encoding

  import codecs
  encoding = codecs.guess_stream_encoding(f)

 2. wrap the stream with a reader for the found encoding

  reader_class = codecs.getreader(encoding)
  g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Jan  8 17:31:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:31:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <loom.20100108T171644-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver <at> palladion.com>
wrote:
> > The BOM should not be seekeable if the file is opened with the proposed
> > "guess encoding from BOM" mode:  it isn't properly part of the stream at
> > all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return?

tell() in the context of text I/O is specified to return an "opaque cookie". So
whatever value it returns would probably be fine, as long as seeking to that
value leaves the file in an acceptable state.

Rewinding (seeking to 0) in the presence of a BOM is already reasonably
supported by the TextIOWrapper object:

>>> dec = codecs.getincrementaldecoder('utf-16')()
>>> dec.decode(b'\xff\xfea\x00b\x00')
'ab'
>>> dec.decode(b'\xff\xfea\x00b\x00')
'\ufeffab'
>>> 
>>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
>>> f = io.TextIOWrapper(bio, encoding='utf-16')
>>> f.read()
'ab'
>>> f.seek(0)
0
>>> f.read()
'ab'

There are tests for this in test_io.py (test_encoded_writes, line 1929, and
test_append_bom and test_seek_bom, line 2045).

Regards

Antoine.




From python at mrabarnett.plus.com  Fri Jan  8 17:47:18 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 16:47:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081127.44044.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <4B476196.4080409@mrabarnett.plus.com>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
> (...)
>> (And yes, I know this happens. Doesn't mean we need to auto-guess by
>> default; there are lots of issues e.g. what should happen after
>> seeking to offset 0?)
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any
>  * fix for seek(0): always ignore the BOM
>  * add an unit test: check that the right encoding is detect, but also the the 
> BOM is ignored (especially after a seek(0))
> 
> BOM encoding doesn't work for writing into a file, so open(filename, "w", 
> encoding="BOM") raises a ValueError.
> 
I think it's similar to universal newline mode. You can tell it that
you're reading UTF-something-encoded text (common forms only).

The preference is UTF-8, but it could be UTF-8-sig (from Windows), or
possibly UTF-16/32, which really need a BOM because there are multiple
bytes per codepoint, so the bytes could be big-endian or little-endian.

The BOM (or signature) tells you what the encoding is, defaulting to
UTF-8 if there's none. If it subsequently raises a DecodeError, then
so be it!

Maybe there should also be a way of determining what encoding it decided
it was, so that you can then write a new file in that same encoding.


From status at bugs.python.org  Fri Jan  8 18:07:13 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jan 2010 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100108170714.014B078669@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/01/10 - 01/08/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2544 open (+27) / 16937 closed (+15) / 19481 total (+42)

Open issues with patches:  1017

Average duration of open issues: 708 days.
Median duration of open issues: 464 days.

Open Issues Breakdown
   open  2509 (+27)
pending    34 ( +0)

Issues Created Or Reopened (43)
_______________________________

Extended slicing with classic class behaves strangely            01/07/10
       http://bugs.python.org/issue7532    reopened mark.dickinson                
       patch                                                                   

optparse library documentation has an insignificant formatting i 01/01/10
CLOSED http://bugs.python.org/issue7618    created  vazovsky                      
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6           01/01/10
CLOSED http://bugs.python.org/issue7619    created  djc                           
                                                                               

Vim syntax highlight                                             01/02/10
       http://bugs.python.org/issue7620    created  july                          
       patch                                                                   

Test issue                                                       01/02/10
CLOSED http://bugs.python.org/issue7621    created  georg.brandl                  
                                                                               

[patch] improve unicode methods: split() rsplit() and replace()  01/03/10
       http://bugs.python.org/issue7622    created  flox                          
       patch                                                                   

PropertyType missing in Lib/types.py                             01/03/10
CLOSED http://bugs.python.org/issue7623    created  wplappert                     
                                                                               

isinstance(... ,collections.Callable) fails with oldstyle class  01/03/10
       http://bugs.python.org/issue7624    created  rgammans                      
                                                                               

bytearray needs more tests for "b.some_method()[0] is not b"     01/03/10
       http://bugs.python.org/issue7625    created  flox                          
       patch                                                                   

Entity references without semicolon in HTMLParser                01/03/10
CLOSED http://bugs.python.org/issue7626    created  stefan.schweizer              
                                                                               

mailbox.MH.remove() lock handling is broken                      01/04/10
       http://bugs.python.org/issue7627    created  sraustein                     
                                                                               

round() doesn't work correctly in 3.1.1                          01/04/10
CLOSED http://bugs.python.org/issue7628    created  bkovt                         
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs" 01/04/10
CLOSED http://bugs.python.org/issue7629    created  popelkopp                     
                                                                               

Strange behaviour of decimal.Decimal                             01/04/10
CLOSED http://bugs.python.org/issue7630    created  parmax                        
                                                                               

undefined label: bltin-file-objects                              01/04/10
CLOSED http://bugs.python.org/issue7631    created  ezio.melotti                  
                                                                               

dtoa.c: oversize b in quorem                                     01/04/10
       http://bugs.python.org/issue7632    created  skrah                         
                                                                               

decimal.py: type conversion in context methods                   01/04/10
       http://bugs.python.org/issue7633    created  skrah                         
       patch, easy                                                             

next/previous links in documentation skip some sections          01/05/10
CLOSED http://bugs.python.org/issue7634    created  gagenellina                   
                                                                               

19.6 xml.dom.pulldom doc: stub?                                  01/05/10
       http://bugs.python.org/issue7635    created  tjreedy                       
                                                                               

Add a set update action to optparse                              01/05/10
CLOSED http://bugs.python.org/issue7636    created  hardkrash                     
       patch                                                                   

Improve 19.5. xml.dom.minidom doc                                01/05/10
       http://bugs.python.org/issue7637    created  tjreedy                       
                                                                               

Counterintuitive str.splitlines() inconsistency.                 01/05/10
CLOSED http://bugs.python.org/issue7638    created  vencabot_teppoo               
                                                                               

bdist_msi fails on files with long names                         01/05/10
       http://bugs.python.org/issue7639    created  mmm77                         
                                                                               

buffered io seek() buggy                                         01/05/10
       http://bugs.python.org/issue7640    created  pakal                         
       patch                                                                   

Built-in Formatter accepts undocumented presentation type        01/06/10
CLOSED http://bugs.python.org/issue7641    created  lrekucki                      
                                                                               

[patch] Minor improvement in os.system doc                       01/06/10
       http://bugs.python.org/issue7642    created  Val                           
       patch                                                                   

What is an ASCII linebreak?                                      01/06/10
       http://bugs.python.org/issue7643    created  flox                          
                                                                               

bug in nntplib.body() method with possible fix                   01/06/10
       http://bugs.python.org/issue7644    created  mdmullins                     
       easy                                                                    

test_distutils fails on Windows XP                               01/06/10
       http://bugs.python.org/issue7645    created  austin987                     
                                                                               

test_telnetlib fails in Windows XP                               01/06/10
       http://bugs.python.org/issue7646    created  austin987                     
                                                                               

Add statvfs flags to the posix module                            01/06/10
       http://bugs.python.org/issue7647    created  ajax at redhat.com               
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                 01/06/10
       http://bugs.python.org/issue7648    created  austin987                     
       easy                                                                    

"u'%c' % char" broken for chars in range '\x80'-'\xFF'           01/07/10
       http://bugs.python.org/issue7649    created  ezio.melotti                  
       patch, easy                                                             

test_uuid is invalid                                             01/07/10
       http://bugs.python.org/issue7650    created  austin987                     
                                                                               

Python3: guess text file charset using the BOM                   01/07/10
       http://bugs.python.org/issue7651    created  haypo                         
       patch                                                                   

Merge C version of decimal into py3k.                            01/07/10
       http://bugs.python.org/issue7652    created  mark.dickinson                
                                                                               

Fix description of the way the PythonPath Windows registry key w 01/07/10
CLOSED http://bugs.python.org/issue7653    created  BoarGules                     
       patch, needs review                                                     

[patch] Enable additional bytes and memoryview tests.            01/07/10
       http://bugs.python.org/issue7654    created  flox                          
       patch                                                                   

PEP 374 Title usage & script redirection typo fixes              01/07/10
CLOSED http://bugs.python.org/issue7655    created  bab9e9                        
       patch                                                                   

test_hashlib fails on some installations (specifically Neal's re 01/08/10
       http://bugs.python.org/issue7656    created  r.david.murray                
                                                                               

test_ctypes failure on AIX 5.3                                   01/08/10
       http://bugs.python.org/issue7657    created  mallayya                      
       patch                                                                   

OS X pythonw.c compile error with 10.4 or earlier deployment tar 01/08/10
       http://bugs.python.org/issue7658    created  ned.deily                     
                                                                               

Problems with attribute assignment on object instances           01/08/10
       http://bugs.python.org/issue7659    created  pakal                         
                                                                               



Issues Now Closed (40)
______________________

shutil fails when copying to NTFS in Linux                        762 days
       http://bugs.python.org/issue1545    benjamin.peterson             
       patch                                                                   

Test                                                              751 days
       http://bugs.python.org/issue1619    georg.brandl                  
                                                                               

Windows Registry issue with 2.5.2 AMD64 msi                       645 days
       http://bugs.python.org/issue2539    loewis                        
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     618 days
       http://bugs.python.org/issue2698    Daniel26                      
                                                                               

optparse print_usage(),.. methods are not documented              542 days
       http://bugs.python.org/issue3340    ezio.melotti                  
                                                                               

reference leaks in test_distutils                                 500 days
       http://bugs.python.org/issue3660    ezio.melotti                  
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                          20 days
       http://bugs.python.org/issue3745    lemburg                       
       26backport                                                              

Add Option to Bind to a Local IP Address in httplib.py            464 days
       http://bugs.python.org/issue3972    gregory.p.smith               
       patch, easy                                                             

no reference for optparse methods                                 388 days
       http://bugs.python.org/issue4635    ezio.melotti                  
                                                                               

PyArg_Parse* should raise TypeError for float parsed with intege  339 days
       http://bugs.python.org/issue5080    mark.dickinson                
       patch                                                                   

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()              282 days
       http://bugs.python.org/issue5576    mark.dickinson                
       patch                                                                   

PyFrame_GetLineNumber                                             244 days
       http://bugs.python.org/issue5954    georg.brandl                  
       patch                                                                   

PyCode_NewEmpty                                                   244 days
       http://bugs.python.org/issue5959    georg.brandl                  
       patch                                                                   

Add non-command help topics to help completion of cmd.Cmd         241 days
       http://bugs.python.org/issue5991    georg.brandl                  
       patch                                                                   

make error                                                        231 days
       http://bugs.python.org/issue6067    georg.brandl                  
                                                                               

no longer possible to hash arrays                                 229 days
       http://bugs.python.org/issue6071    benjamin.peterson             
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           173 days
       http://bugs.python.org/issue6511    r.david.murray                
                                                                               

use different mechanism for pythonw on osx                        127 days
       http://bugs.python.org/issue6834    ned.deily                     
       easy                                                                    

Py3k doc: "from __future__ import division" not necessary          32 days
       http://bugs.python.org/issue7432    ezio.melotti                  
                                                                               

cPickle: stack underflow in load_pop()                             31 days
       http://bugs.python.org/issue7455    pitrou                        
       patch                                                                   

crash in str.rfind() with an invalid start value                   25 days
       http://bugs.python.org/issue7458    pitrou                        
       patch                                                                   

Implement fastsearch algorithm for rfind/rindex                    26 days
       http://bugs.python.org/issue7462    ezio.melotti                  
       patch                                                                   

GZipFile.readline too slow                                         20 days
       http://bugs.python.org/issue7471    pitrou                        
       patch                                                                   

ssl module documentation: SSLSocket.unwrap description shown twi    5 days
       http://bugs.python.org/issue7592    georg.brandl                  
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            7 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

optparse library documentation has an insignificant formatting i    2 days
       http://bugs.python.org/issue7618    ezio.melotti                  
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6              1 days
       http://bugs.python.org/issue7619    djc                           
                                                                               

Test issue                                                          0 days
       http://bugs.python.org/issue7621    ezio.melotti                  
                                                                               

PropertyType missing in Lib/types.py                                0 days
       http://bugs.python.org/issue7623    benjamin.peterson             
                                                                               

Entity references without semicolon in HTMLParser                   2 days
       http://bugs.python.org/issue7626    r.david.murray                
                                                                               

round() doesn't work correctly in 3.1.1                             0 days
       http://bugs.python.org/issue7628    benjamin.peterson             
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs"    0 days
       http://bugs.python.org/issue7629    r.david.murray                
                                                                               

Strange behaviour of decimal.Decimal                                0 days
       http://bugs.python.org/issue7630    mark.dickinson                
                                                                               

undefined label: bltin-file-objects                                 0 days
       http://bugs.python.org/issue7631    pitrou                        
                                                                               

next/previous links in documentation skip some sections             0 days
       http://bugs.python.org/issue7634    georg.brandl                  
                                                                               

Add a set update action to optparse                                 3 days
       http://bugs.python.org/issue7636    r.david.murray                
       patch                                                                   

Counterintuitive str.splitlines() inconsistency.                    0 days
       http://bugs.python.org/issue7638    flox                          
                                                                               

Built-in Formatter accepts undocumented presentation type           0 days
       http://bugs.python.org/issue7641    eric.smith                    
                                                                               

Fix description of the way the PythonPath Windows registry key w    0 days
       http://bugs.python.org/issue7653    r.david.murray                
       patch, needs review                                                     

PEP 374 Title usage & script redirection typo fixes                 0 days
       http://bugs.python.org/issue7655    georg.brandl                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 22 [patch] improve unicode methods: split() rsplit() and replace()    5 days
open    http://bugs.python.org/issue7622   

 14 Test suite emits many DeprecationWarnings when -3 is enabled      91 days
open    http://bugs.python.org/issue7092   

 13 unicode_escape codec does not escape quotes                        8 days
open    http://bugs.python.org/issue7615   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    0 days
open    http://bugs.python.org/issue7658   

  8 Merge C version of decimal into py3k.                              1 days
open    http://bugs.python.org/issue7652   

  8 "u'%c' % char" broken for chars in range '\x80'-'\xFF'             2 days
open    http://bugs.python.org/issue7649   

  8 decimal.py: type conversion in context methods                     4 days
open    http://bugs.python.org/issue7633   

  8 Patch for [ 735515 ] urllib2 should cache 301 redir              906 days
open    http://bugs.python.org/issue1755841

  7 Test issue                                                         0 days
closed  http://bugs.python.org/issue7621   

  7 Cannot use both read and readline method in same ZipExtFile obj    8 days
open    http://bugs.python.org/issue7610   




From tseaver at palladion.com  Fri Jan  8 22:09:54 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:09:54 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <hi86v3$3j5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> The BOM should not be seekeable if the file is opened with the proposed
>> "guess encoding from BOM" mode:  it isn't properly part of the stream at
>> all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return? And regardless of that, .seek(0) should put the file in that
> same initial state.

I think the behavior should be something like:

 >>> f = open('/path/to/maybe-BOM-encoded-file', 'r', encoding='BOM')
 >>> f.tell()
 0L
 >>> f.seek(-1)
 >>> f.tell() # count of unicode chars in decoded stream
 45L
 >>> f.seek(0)
 >>> f.read(1) # read first unicode char decoded from stream.
 'A'

In other words, the BOM is not readable / seekable at all:  it is
invisible to the consumer of the decoded stream.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHnyIACgkQ+gerLs4ltQ6s3QCgznD+7FbUzfCbe5TS6OcoXjMg
rdgAoJAMEXe2xwLCIwJaZ6XA6rVyTIAi
=oXb3
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:19:10 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:19:10 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B475C72.1010207@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com>
Message-ID: <hi87gg$8ge$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:

> Shouldn't this encoding guessing be a separate function that you call
> on either a file or a seekable stream ?
> 
> After all, detecting encodings is just as useful to have for non-file
> streams.

Other stream sources typically have out-of-band ways to signal the
encoding:  only when reading from the filesystem do we pretty much
*have* to guess, and in that case the BOM / signature is the best
heuristic we have.  Also, some non-file streams are not seekable, and so
can't be guessed via a pre-pass.

> You'd then avoid having to stuff everything into
> a single function call and also open up the door for more complex
> application specific guess work or defaults.
> 
> The whole process would then have two steps:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_file_encoding(filename)

Filename is not enough information:  or do you mean that API to actually
open the stream?

>  2. open the file with the found encoding
> 
>   f = open(filename, encoding=encoding)
> 
> For seekable streams f, you'd have:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_stream_encoding(f)
> 
>  2. wrap the stream with a reader for the found encoding
> 
>   reader_class = codecs.getreader(encoding)
>   g = reader_class(f)
> 


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoU4ACgkQ+gerLs4ltQ5o3QCeLOJ7J91E+5f66vhgu1BUhYh4
9UgAnR2IeCd0BCsPez8ZilGNHJfhRn3Y
=SoPb
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:14:59 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:14:59 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <hi878l$7os$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:

>>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that.  That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
> 
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.

If the programmer opens the file using a "guess using the BOM" encoding,
 Python should *not* attempt to verify that the file is properly
encoded:  it should check for (and consume) any BOM, and then return a
stream which uses the encoding inferred from the BOM.  Any errors should
be handled later, when characters are read, exactly as if the file had
been opened with the same encoding guessed from the BOM.

> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
> 
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Agreed.  Having that marker at the start of the file makes interop with
other tools *much* easier.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoFMACgkQ+gerLs4ltQ73dACffwUfyh6Q9vUnKYf367QFjNcU
RRMAoNuKCWEx7j+MSdTv+UjhAPynBc14
=uAX6
-----END PGP SIGNATURE-----



From eric at trueblade.com  Fri Jan  8 22:40:47 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 8 Jan 2010 16:40:47 -0500 (EST)
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
Message-ID: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>

>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
>>
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
>
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

But what if the file were in (for example) a zip file? I think you
definitely want to have access to this functionality outside of open().

Eric.



From foom at fuhm.net  Fri Jan  8 22:49:23 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 8 Jan 2010 16:49:23 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de> <hi878l$7os$1@ger.gmane.org>
Message-ID: <D481E750-3EE7-4498-9959-B4E34E02FFC6@fuhm.net>

On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote:
>> I understood this proposal as a general processing guideline, not
>> something the io library should do (but, say, a text editor).
>>
>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly  
>> have
>> a byte order - hence I call it a signature, not the BOM. As a  
>> signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote  
>> the
>> encoding is useful and helps reducing moji-bake. This is the reason  
>> it's
>> used on Windows: notepad would normally assume that text is in the  
>> ANSI
>> code page, and for compatibility, it can't stop doing that. So the  
>> UTF-8
>> signature gives them an exit strategy.
>
> Agreed.  Having that marker at the start of the file makes interop  
> with
> other tools *much* easier.

Putting the BOM at the beginning of UTF-8 text files is not a good  
idea, it makes interop much *worse* on a unix system, not better.  
Without the BOM, most commands do the right thing with UTF-8 text.  
E.g. to concatenate two files:

$ cat file-1 file-2 > file-3

With a BOM at the beginning of the file, it won't work right. Of  
course, you could modify "cat" (and every other stream processing  
command) to know how to consume and emit BOMs, and omit the extra one  
that would show up in the middle of the stream...but even that can't  
work; what about:
$ (cat file-1; cat file-2) > file-3.

Should the shell now know that when you run multiple commands, it  
should eat the BOM emitted from the second command?

Basically, using a BOM in a utf-8 file is just not a good idea: it  
completely ruins interop with every standard unix tool.

This is not to say that Python shouldn't have a way to read a file  
with a UTF-8 BOM: it just shouldn't encourage you to *write* such files.

James


From mal at egenix.com  Fri Jan  8 22:51:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 22:51:26 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>	<4B475C72.1010207@egenix.com>
	<hi87gg$8ge$1@ger.gmane.org>
Message-ID: <4B47A8DE.1080401@egenix.com>

Tres Seaver wrote:
> M.-A. Lemburg wrote:
> 
>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
> 
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
> 
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

Sure there are non-seekable file streams, but at least when
using StringIO-type streams you don't have that restriction.

An encoding detection function would provide more use in other
cases as well, so instead of hiding away the functionality in
the open() constructor, I'm suggesting to make expose it via
the codecs module.

Applications can then use it where necessary and also provide their
own defaults, using other heuristics.

>> You'd then avoid having to stuff everything into
>> a single function call and also open up the door for more complex
>> application specific guess work or defaults.
> 
>> The whole process would then have two steps:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_file_encoding(filename)
> 
> Filename is not enough information:  or do you mean that API to actually
> open the stream?

Yes. The API would open the file, guess the encoding and then
close it again. If you don't want that to happen, you could use
the second API I mentioned below on the already open file.

Note that this function could detect a lot more encodings with
reasonably high probability than just BOM-prefixed ones,
e.g. we could also add support to detect encoding declarations
such as the ones we use in Python source files.

>>  2. open the file with the found encoding
> 
>>   f = open(filename, encoding=encoding)
> 
>> For seekable streams f, you'd have:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_stream_encoding(f)

I forgot to mention: This API needs to position the file pointer
to the start of the first data byte.

>>  2. wrap the stream with a reader for the found encoding
> 
>>   reader_class = codecs.getreader(encoding)
>>   g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From tseaver at palladion.com  Fri Jan  8 22:59:04 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:59:04 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <hi89r8$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
>>> Shouldn't this encoding guessing be a separate function that you call
>>> on either a file or a seekable stream ?
>>>
>>> After all, detecting encodings is just as useful to have for non-file
>>> streams.
>> Other stream sources typically have out-of-band ways to signal the
>> encoding:  only when reading from the filesystem do we pretty much
>> *have* to guess, and in that case the BOM / signature is the best
>> heuristic we have.  Also, some non-file streams are not seekable, and so
>> can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

If the application expects a possibly-BOM-signature-marked file, but you
pass it mismatched garbage:

  >>> f = open('some.zip', encoding='BOM")

the error handling should be the same as if you passed any other
mismatched encoding:

  >>> f = open('some.zip', encoding='UTF8')

i.e., you discover the error when you try to read from the (non)encoded
stream, not when you open it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHqpwACgkQ+gerLs4ltQ7uAACeKEc+WT4TASGcVl1Hfqe6L9La
I6EAn1pJtngtLWPdothGbYB+zUabEvTW
=TjBK
-----END PGP SIGNATURE-----



From victor.stinner at haypocalc.com  Fri Jan  8 23:10:32 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 23:10:32 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <201001082310.33029.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 22:40:47, Eric Smith a ?crit :
> >> Shouldn't this encoding guessing be a separate function that you call
> >> on either a file or a seekable stream ?
> >>
> >> After all, detecting encodings is just as useful to have for non-file
> >> streams.
> >
> > Other stream sources typically have out-of-band ways to signal the
> > encoding:  only when reading from the filesystem do we pretty much
> > *have* to guess, and in that case the BOM / signature is the best
> > heuristic we have.  Also, some non-file streams are not seekable, and so
> > can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

FYI my patch (encoding="BOM") is implemented in TextIOWrapper, and 
TextIOWrapper takes a binary stream as input, not a filename.

-- 
Victor Stinner
http://www.haypocalc.com/


From yoann.padioleau at facebook.com  Fri Jan  8 23:59:52 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Fri, 8 Jan 2010 14:59:52 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464F1C.7020404@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
	<4B464F1C.7020404@v.loewis.de>
Message-ID: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>


On Jan 7, 2010, at 1:16 PM, Martin v. L?wis wrote:

>>> astgen.py is not used to process asdl files; ast.txt lives right
>>> next to astgen.py. Instead, the asdl file is processed by
>>> Parser/asdl_c.py.
>> 
>> Yes, I know that. That's why I asked about the relation between
>> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
>> expose as a reflection mechanism things that were generated from
>> ast.txt, then there could be a mismatch. Where does ast.txt comes
>> from ? Shouldn't it be generated itself from Python.adsl ?
> 
> What you may not be aware of is that Tools/compiler (and the
> compiler package that it builds on) are both unused and unmaintained.

I see. So if people want to analyze python code they have to use
other tools (like rope?) ? or use reflection ?

> 
> If the package stops working correctly - tough luck.
> 
>> So we would have
>> 
>> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
>> containing all the UnarySub, Expression, classes that represents a
>> Python AST.
> 
> No - what actually happens in Python 3.x is this: both the compiler
> package and Tools/compiler are removed.

Ok. I will then create my own ast classes generator.

Thanks.


> 
> Regards,
> Martin



From g.brandl at gmx.net  Sat Jan  9 00:10:24 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 09 Jan 2010 00:10:24 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<4B46F54D.9090103@v.loewis.de>
	<hi878l$7os$1@ger.gmane.org>
Message-ID: <hi8e1n$sos$1@ger.gmane.org>

Am 08.01.2010 22:14, schrieb Tres Seaver:

>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly have
>> a byte order - hence I call it a signature, not the BOM. As a signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote the
>> encoding is useful and helps reducing moji-bake. This is the reason it's
>> used on Windows: notepad would normally assume that text is in the ANSI
>> code page, and for compatibility, it can't stop doing that. So the UTF-8
>> signature gives them an exit strategy.
> 
> Agreed.  Having that marker at the start of the file makes interop with
> other tools *much* easier.

Except if only 50% of the other tools support the signature.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From martin at v.loewis.de  Sat Jan  9 00:57:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 00:57:46 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>	<4B464499.4020305@v.loewis.de>	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>	<4B464F1C.7020404@v.loewis.de>
	<F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
Message-ID: <4B47C67A.3020302@v.loewis.de>

> I see. So if people want to analyze python code they have to use
> other tools (like rope?) ? or use reflection ?

Correct. One such tool might be the true Python compiler, along
with the _ast module.

Regards,
Martin


From victor.stinner at haypocalc.com  Sat Jan  9 00:59:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 00:59:00 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
Message-ID: <201001090059.00848.victor.stinner@haypocalc.com>

Hi,

Thanks for all the answers! I will try to sum up all ideas here.


(1) Change default open() behaviour or make it optional?

Guido would like to add an option and keep open() unchanged. He wrote that 
checking for BOM and using system locale are too much different to be the same 
option (encoding=None).

Antoine would like to check BOM by default, because both options (system 
locale vs checking for BOM) is the same thing.

About Antoine choice (encoding=None): which file modes would check for a BOM? 
I would like to answer only the read only mode, but then open(filename, "r") 
and open(filename, "r+") would behave differently?

  => 1 point for Guido (encoding="BOM" is more explicit)

Antoine choice has the advantage of directly support UTF-8+BOM, UTF-16 and 
UTF-32 for all applications and all modules using open(filename).

  => 1 point for Antoine


(2) Check for a BOM while reading or detect it before?

Everybody agree that checking BOM is an interesting option and should not be 
limited to open().

Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
name or a binary file-like object: it returns the encoding and seek to the 
file start or just after the BOM.

I dislike this function because it requires extra file operations (open 
(optional), read() and seek()) and it doesn't work if the file is not seekable 
(eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
avoid extra file operations.

Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
any file-like object.


(3) tell() and seek() on a text file starting with a BOM

To be consistent with Antoine example:

   >>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
   >>> f = io.TextIOWrapper(bio, encoding='utf-16')
   >>> f.read()
   'ab'
   >>> f.seek(0)
   0
   >>> f.read()
   'ab'

TextIOWrapper:

 * tell() should return zero at file start,
 * seek(0) should go be to file start,
 * and the BOM should always be "ignored".

I mean:

  with open("utf8bom.txt", encoding="BOM") as fp:
     assert fp.tell() == 0   
     text = fp.read() # no BOM here
     fp.seek(0)
     assert fp.read() == text

--

About my patch:

 - BOM check is explicit: open(filebame,  encoding="BOM")
 - tell() / seek(0) works as expected
 - BOM bytes are always skipped in read() / readlines() result

Hum, I don't know if this email can be called a sum up ;-)

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 01:09:48 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 00:09:48 +0000 (UTC)
Subject: [Python-Dev] Quick sum up about open() + BOM
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <loom.20100109T010657-648@post.gmane.org>

Hello Victor,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> (1) Change default open() behaviour or make it optional?
> 
[...]
> 
> Antoine would like to check BOM by default, because both options (system 
> locale vs checking for BOM) is the same thing.

To be clear, I am not saying it is the same thing. What I think is that it would
be a mistake to use a mildly unreliable heuristic by default (the locale +
device encoding heuristic) but refuse to trust a more reliable heuristic (the
BOM-based detection algorithm).

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Sat Jan  9 01:14:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 00:14:39 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <loom.20100109T010657-648@post.gmane.org>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<loom.20100109T010657-648@post.gmane.org>
Message-ID: <4B47CA6F.5000607@voidspace.org.uk>

On 09/01/2010 00:09, Antoine Pitrou wrote:
> Hello Victor,
>
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>    
>> (1) Change default open() behaviour or make it optional?
>>
>>      
> [...]
>    
>> Antoine would like to check BOM by default, because both options (system
>> locale vs checking for BOM) is the same thing.
>>      
> To be clear, I am not saying it is the same thing. What I think is that it would
> be a mistake to use a mildly unreliable heuristic by default (the locale +
> device encoding heuristic) but refuse to trust a more reliable heuristic (the
> BOM-based detection algorithm).
>    

I concur. On Windows both UTF-8 and signature are very common, yet the 
platform default is the truly awful CP1252.

All the best,

Michael
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From floris.bruynooghe at gmail.com  Sat Jan  9 01:26:05 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 9 Jan 2010 00:26:05 +0000
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B46F6D7.1050301@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
	<4B46F6D7.1050301@v.loewis.de>
Message-ID: <20100109002605.GB13536@laurie.devork>

On Fri, Jan 08, 2010 at 10:11:51AM +0100, "Martin v. L?wis" wrote:
> Nicholas Bastin wrote:
> > I think this problem probably needs to move over to distutils-sig, as
> > it doesn't seem to be specific to the way that Python itself uses
> > distutils.
> 
> I'm fairly skeptical that anybody on distutils SIG is interested in
> details of the Python build process...

Uh, hum.  Unfounded skepticism.  ;-)
But as said filing a bug sounds better in this case.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From v+python at g.nevcal.com  Sat Jan  9 01:47:38 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 16:47:38 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B47D22A.8070305@g.nevcal.com>

On approximately 1/8/2010 3:59 PM, came the following characters from 
the keyboard of Victor Stinner:
> Hi,
>
> Thanks for all the answers! I will try to sum up all ideas here.

One concern I have with this implementation encoding="BOM" is that if 
there is no BOM it assumes UTF-8.  That is probably a good assumption in 
some circumstances, but not in others.

* It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
(cases where the endianness is unspecified) contain a BOM.  Hence, it 
might be that someone would expect a UTF-16LE (or any of the formats 
that don't require a BOM, rather than UTF-8), but be willing to accept 
any BOM-discriminated format.

* Potentially, this could be expanded beyond the various Unicode 
encodings... one could envision that a program whose data files 
historically were in any particular national language locale, could want 
to be enhance to accept Unicode, and could declare that they will accept 
any BOM-discriminated format, but want to default, in the absence of a 
BOM, to the original national language locale that they historically 
accepted.  That would provide a migration path for their old data files.

So the point is, that it might be nice to have 
"BOM-otherEncodingForDefault" for each other encoding that Python 
supports.  Not sure that is the right API, but I think it is expressive 
enough to handle the cases above.  Whether the cases solve actual 
problems or not, I couldn't say, but they seem like reasonable cases.

It would, of course, be nicest if OS metadata had been invented way back 
when, for all OSes, such that all text files were flagged with their 
encoding... then languages could just read the encoding and do the right 
thing! But we live in the real world, instead.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From python at mrabarnett.plus.com  Sat Jan  9 02:12:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 09 Jan 2010 01:12:28 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B47D7FC.4070703@mrabarnett.plus.com>

Glenn Linderman wrote:
> On approximately 1/8/2010 3:59 PM, came the following characters from 
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if 
> there is no BOM it assumes UTF-8.  That is probably a good assumption in 
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
> (cases where the endianness is unspecified) contain a BOM.  Hence, it 
> might be that someone would expect a UTF-16LE (or any of the formats 
> that don't require a BOM, rather than UTF-8), but be willing to accept 
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode 
> encodings... one could envision that a program whose data files 
> historically were in any particular national language locale, could want 
> to be enhance to accept Unicode, and could declare that they will accept 
> any BOM-discriminated format, but want to default, in the absence of a 
> BOM, to the original national language locale that they historically 
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have 
> "BOM-otherEncodingForDefault" for each other encoding that Python 
> supports.  Not sure that is the right API, but I think it is expressive 
> enough to handle the cases above.  Whether the cases solve actual 
> problems or not, I couldn't say, but they seem like reasonable cases.
> 
> It would, of course, be nicest if OS metadata had been invented way back 
> when, for all OSes, such that all text files were flagged with their 
> encoding... then languages could just read the encoding and do the right 
> thing! But we live in the real world, instead.
> 
What about listing the possible encodings? It would try each in turn
until it found one where the BOM matched or had no BOM:

     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')

or is that taking it too far?


From martin at v.loewis.de  Sat Jan  9 02:23:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 02:23:07 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47CA6F.5000607@voidspace.org.uk>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<loom.20100109T010657-648@post.gmane.org>
	<4B47CA6F.5000607@voidspace.org.uk>
Message-ID: <4B47DA7B.6050505@v.loewis.de>

>>> Antoine would like to check BOM by default, because both options
>>> (system locale vs checking for BOM) is the same thing.
>>> 
>> To be clear, I am not saying it is the same thing. What I think is 
>> that it would be a mistake to use a mildly unreliable heuristic by
>> default (the locale + device encoding heuristic) but refuse to
>> trust a more reliable heuristic (the BOM-based detection
>> algorithm).
>> 
> 
> I concur. On Windows both UTF-8 and signature are very common, yet
> the platform default is the truly awful CP1252.

While I would support combining BOM detection in the case where a file
is opened for reading and no encoding is specified, I see two problems:
a) if a seek operations is performed before having looked at the BOM,
   no determination would have been made
b) what encoding should it use on writing?

Regards,
Martin



From v+python at g.nevcal.com  Sat Jan  9 02:49:12 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 17:49:12 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <4B47E098.6030703@g.nevcal.com>

On approximately 1/8/2010 5:12 PM, came the following characters from 
the keyboard of MRAB:
> Glenn Linderman wrote:
>> On approximately 1/8/2010 3:59 PM, came the following characters from 
>> the keyboard of Victor Stinner:
>>> Hi,
>>>
>>> Thanks for all the answers! I will try to sum up all ideas here.
>>
>> One concern I have with this implementation encoding="BOM" is that if 
>> there is no BOM it assumes UTF-8.  That is probably a good assumption 
>> in some circumstances, but not in others.
>>
>> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
>> encoded files include a BOM.  It is only required that UTF-16 and 
>> UTF-32 (cases where the endianness is unspecified) contain a BOM.  
>> Hence, it might be that someone would expect a UTF-16LE (or any of 
>> the formats that don't require a BOM, rather than UTF-8), but be 
>> willing to accept any BOM-discriminated format.
>>
>> * Potentially, this could be expanded beyond the various Unicode 
>> encodings... one could envision that a program whose data files 
>> historically were in any particular national language locale, could 
>> want to be enhance to accept Unicode, and could declare that they 
>> will accept any BOM-discriminated format, but want to default, in the 
>> absence of a BOM, to the original national language locale that they 
>> historically accepted.  That would provide a migration path for their 
>> old data files.
>>
>> So the point is, that it might be nice to have 
>> "BOM-otherEncodingForDefault" for each other encoding that Python 
>> supports.  Not sure that is the right API, but I think it is 
>> expressive enough to handle the cases above.  Whether the cases solve 
>> actual problems or not, I couldn't say, but they seem like reasonable 
>> cases.
>>
>> It would, of course, be nicest if OS metadata had been invented way 
>> back when, for all OSes, such that all text files were flagged with 
>> their encoding... then languages could just read the encoding and do 
>> the right thing! But we live in the real world, instead.
>>
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
>
>     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

That sounds very flexible -- but in net effect it would only make 
illegal a subset of the BOM-containing encodings (those not listed) 
without making legal any additional encodings other than the non-BOM 
encoding.  Whether prohibiting a subset of BOM-containing encodings is a 
useful use case, I couldn't say... but my goal would be to included as 
many different file encodings on input as possible: without a BOM, that 
is exactly 1 (unless there are other heuristics), with a BOM, it is 
1+all-BOM-containing encodings.  Your scheme would permit numbers of 
encodings accepted to vary between 1 and 1+all-BOM-containing encodings.

(I think everyone can agree there are 5 different byte sequences that 
can be called a Unicode BOM.  The likelihood of them appearing in any 
other text encoding created by mankind depends on those other encodings 
-- but it is not impossible.  It is truly up to the application to 
decide whether BOM detection could potentially conflict with files in 
some other encoding that would be acceptable to the application.)

So I think it is taking it further than I can see value in, but I'm 
willing to be convinced otherwise.  I see only a need for detecting BOM, 
and specifying a default encoding to be used if there is no BOM.  Note 
that it might be nice to have a specification for using current 
encoding=None heuristic -- perhaps encoding="BOM-None" per my originally 
proposed syntax.  But I'm still not saying that is the best syntax.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From ncoghlan at gmail.com  Sat Jan  9 04:07:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Jan 2010 13:07:12 +1000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B476196.4080409@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>
	<4B476196.4080409@mrabarnett.plus.com>
Message-ID: <4B47F2E0.7090403@gmail.com>

MRAB wrote:
> Maybe there should also be a way of determining what encoding it decided
> it was, so that you can then write a new file in that same encoding.

I thought of that question as well - the f.encoding attribute on the
opened file should be sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Sat Jan  9 06:48:36 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 06:48:36 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47E098.6030703@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com> <4B47D7FC.4070703@mrabarnett.plus.com>
	<4B47E098.6030703@g.nevcal.com>
Message-ID: <319e029f1001082148h5cee2c0bn76f70a17766ca69e@mail.gmail.com>

It seems to me that when opening a file, the following is the only
flow that makes sense for the typical opening of a file flow:

if encoding is not None:
   use encoding
elif file has BOM:
   use BOM
else:
   use system default

And hence a encoding='BOM' isn't needed there. Although I'm trying to
come up with usecases that doesn't work with this, I can't. :)

BUT

When writing things are not so easy though. Apparently some encodings
require a BOM to be written, but others do not, but allow it, and some
has no byte order mark. So there you have to be able to write the BOM,
or not. And that's either a new parameter, because you can't use
encoding='BOM' since you need to specify the encoding as well, or a
new method.

I would suggest a BOM parameter, and maybe a method as  well.

BOM=None|True|False

Where "None" means a sane default behaviour, that is write a BOM if
the encoding require it.
"True" means write a BOM if the encoding *supports* it.
"False" means Don't write a BOM even if the encoding requires it
(because I know what I'm doing)

if 'w' in mode: # But not 'r' or 'a'
    if BOM == True and encoding in (ENCODINGS THAT ALLOW BOM):
        write_bom = True
    elif BOM == False:
       write_bom = False
    elif BOM == None and encoding in (ENCODINGS THAT REQUIRE BOM):
          write_bom = True
    else:
          write_bom = False
else:
    write_bom = False

For reading this parameter could either be a noop, or possibly change
the behavior somehow, if a usecase where that makes sense can be
imagined.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From walter at livinglogic.de  Sat Jan  9 11:51:57 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 11:51:57 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B485FCD.2040901@livinglogic.de>

On 09.01.10 01:47, Glenn Linderman wrote:

> On approximately 1/8/2010 3:59 PM, came the following characters from
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.  That is probably a good assumption in
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32
> (cases where the endianness is unspecified) contain a BOM.  Hence, it
> might be that someone would expect a UTF-16LE (or any of the formats
> that don't require a BOM, rather than UTF-8), but be willing to accept
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode
> encodings... one could envision that a program whose data files
> historically were in any particular national language locale, could want
> to be enhance to accept Unicode, and could declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have
> "BOM-otherEncodingForDefault" for each other encoding that Python
> supports.  Not sure that is the right API, but I think it is expressive
> enough to handle the cases above.  Whether the cases solve actual
> problems or not, I couldn't say, but they seem like reasonable cases.

This is doable with the currect API. Simply define a codec search
function that handles all encoding names that start with "BOM-" and pass
the "otherEncodingForDefault" part along to the codec.

> It would, of course, be nicest if OS metadata had been invented way back
> when, for all OSes, such that all text files were flagged with their
> encoding... then languages could just read the encoding and do the right
> thing! But we live in the real world, instead.

Servus,
   Walter


From walter at livinglogic.de  Sat Jan  9 12:18:33 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 12:18:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081140.28124.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
Message-ID: <4B486609.2050804@livinglogic.de>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
>>> Builtin open() function is unable to open an UTF-16/32 file starting with
>>> a BOM if the encoding is not specified (raise an unicode error). For an
>>> UTF-8 file starting with a BOM, read()/readline() returns also the BOM
>>> whereas the BOM should be "ignored".
>> It depends. If you use the utf-8-sig encoding, it *will* ignore the
>> UTF-8 signature.
> 
> Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
> UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
> remove the BOM after the first read (much harder if you use a module like 
> ConfigParser or csv).
> 
>>> Since my proposition changes the result TextIOWrapper.read()/readline()
>>> for files starting with a BOM, we might introduce an option to open() to
>>> enable the new behaviour. But is it really needed to keep the backward
>>> compatibility?
>> Absolutely. And there is no need to produce a new option, but instead
>> use the existing options: define an encoding that auto-detects the
>> encoding from the family of BOMs. Maybe you call it encoding="sniff".
> 
> Good idea, I choosed open(filename, encoding="BOM").

On the surface this looks like there's an encoding named "BOM", but 
looking at your patch I found that the check is still done in 
TextIOWrapper. IMHO the best approach would to the implement a *real* 
codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
the IO library. It could even be developed as a standalone project and 
published in the Cheeseshop.

To see how something like this can be done, take a look at the UTF-16 
codec, that switches to bigendian or littleendian mode depending on the 
first read/decode call.

Servus,
    Walter







From victor.stinner at haypocalc.com  Sat Jan  9 13:37:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:37:06 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47DA7B.6050505@v.loewis.de>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47CA6F.5000607@voidspace.org.uk> <4B47DA7B.6050505@v.loewis.de>
Message-ID: <201001091337.06596.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:23:07, Martin v. L?wis a ?crit :
> While I would support combining BOM detection in the case where a file
> is opened for reading and no encoding is specified, I see two problems:
> a) if a seek operations is performed before having looked at the BOM,
>    no determination would have been made

TextIOWrapper doesn't support seek to an arbitrary byte. It uses "cookie" 
which is an opaque value. Reuse a cookie from another file or an old cookie is 
forbidden (but it doesn't raise an error). This is not specific to the BOM 
checking: the problem already exist for encodings using a BOM (eg. UTF-16).

> b) what encoding should it use on writing?

Don't change anything to writing.

With Antoince choice: open('file.txt', 'w', encoding=None) continue to use the 
actual heuristic (os.device_encoding() or system locale).

With Guido choice, encoding="BOM": it raises an error, because BOM check is 
not supported when writing into a file. How could the BOM be checked when 
creating a new (empty) file!?

-- 
Victor Stinner
http://www.haypocalc.com/


From mal at egenix.com  Sat Jan  9 13:45:58 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 09 Jan 2010 13:45:58 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B487A86.4020603@egenix.com>

Victor Stinner wrote:
> (2) Check for a BOM while reading or detect it before?
> 
> Everybody agree that checking BOM is an interesting option and should not be 
> limited to open().
> 
> Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
> name or a binary file-like object: it returns the encoding and seek to the 
> file start or just after the BOM.
> 
> I dislike this function because it requires extra file operations (open 
> (optional), read() and seek()) and it doesn't work if the file is not seekable 
> (eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
> avoid extra file operations.
> 
> Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
> any file-like object.

Yes, but the implementation is limited to just BOM checking
and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

With a codecs module function we could easily extend the
encoding detection to more file types, e.g. XML files,
Python source code files, etc. that use other mechanisms
for defining the encoding.

BTW: I haven't looked at your implementation, but what happens
when your BOM check fails ? Will the implementation add the
already read bytes back to a buffer ?

This rollback action is the only reason for needing a
seekable stream in codecs.guess_stream_encoding().

Another point to consider:

AFAIK, we currently have a moratorium on changes to Python
builtins. How does that match up with the proposed changes ?

Using a new codec like Walter suggested would move the
implementation into the stdlib for which doesn't the
moratorium doesn't apply.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From victor.stinner at haypocalc.com  Sat Jan  9 13:54:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:54:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <201001091354.17239.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 01:47:38, vous avez ?crit :
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.

If no BOM is found, it fallback to the current heuristic: os.device_encoding() 
or system local.

> (...) Hence, it might be that someone would expect a UTF-16LE (or any of 
> the formats that don't require a BOM, rather than UTF-8), but be willing 
> to accept any BOM-discriminated format.
> (...) declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted

You mean "if there is a BOM, use it, otherwise fallback to a specific 
charset"? How could it be declared? Maybe:

   open("file.txt", check_bom=True, encoding="UTF16-LE")
   open("file.txt", check_bom=True, encoding="latin1")

About falling back to UTF-8, it would be written:

   open("file.txt", check_bom=True, encoding="UTF-8")

As explained before, check_bom=True is only accepted for read only file mode.

Well, why not. This is a third choice for my point (1) :-) It's between Guido 
and Antoine choice, and I like it because we can fallback to UTF-8 instead of 
the dummy system locale: Windows users will be happy to be able to use UTF-8 
:-) I prefer to fallback to a fixed encoding then depending on the system 
locale.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:34:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:34:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <201001091434.17521.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:12:28, MRAB a ?crit :
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
> 
>      my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

Yes, you're taking it foo far :-) Checking BOM is reliable, whereas *guessing* 
the charset only using the byte stream can only be an heuristic. Guess a 
charset is a complex problem, they are 3rd party library to do that, like the 
chardet project.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:38:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:38:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B486609.2050804@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <201001091438.43576.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
> > Good idea, I choosed open(filename, encoding="BOM").
> 
> On the surface this looks like there's an encoding named "BOM", but
> looking at your patch I found that the check is still done in
> TextIOWrapper. IMHO the best approach would to the implement a *real*
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
> the IO library. It could even be developed as a standalone project and
> published in the Cheeseshop.

Why not, this is another solution to the point (2) (Check for a BOM while 
reading or detect it before?). Which encoding would be used if there is not 
BOM? UTF-8 sounds like a good choice.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:50:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:50:28 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B487A86.4020603@egenix.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B487A86.4020603@egenix.com>
Message-ID: <201001091450.28497.victor.stinner@haypocalc.com>

Hi,

Le samedi 09 janvier 2010 13:45:58, vous avez ?crit :
> > Note: I implemented the BOM check in TextIOWrapper; so it's already
> > usable for any file-like object.
> 
> Yes, but the implementation is limited to just BOM checking
> and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

Sure, but that's already better than no BOM check :-) It looks like many 
people would apprecite UTF-8-SIG detection, since this encoding is common on 
Windows.

> BTW: I haven't looked at your implementation, but what happens
> when your BOM check fails ? Will the implementation add the
> already read bytes back to a buffer ?

My implementation is done between buffer.read() and decoder.decode(data). If 
there is a BOM: set the encoding and remove the BOM bytes from the byte 
string. Otherwise, use another algorithm to choose the encoding and leave the 
byte string unchanged.

It can be seen as a codec: it works like UTF-16 and UTF-32 codecs ;-)

> AFAIK, we currently have a moratorium on changes to Python
> builtins. How does that match up with the proposed changes ?

Oh yes, I forgot the moratorium. In all solutions, some of them don't change 
the API. Eg. Antoine proposed to leave the API unchanged: open(file) => 
open(file) :-) I don't know if it's compatible with the moratorium or not.

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 16:05:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 15:05:45 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <loom.20100109T160248-501@post.gmane.org>

Walter D?rwald <walter <at> livinglogic.de> writes:
> 
> On the surface this looks like there's an encoding named "BOM", but 
> looking at your patch I found that the check is still done in 
> TextIOWrapper. IMHO the best approach would to the implement a *real* 
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
> the IO library. It could even be developed as a standalone project and 
> published in the Cheeseshop.

Sorry but this is missing the point. The point here is to improve the open()
function. I'm sure people who know about encodings are able to install the
chardet library or even whip up their own BOM detection routine...


Antoine.




From benjamin at python.org  Sat Jan  9 18:29:33 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 11:29:33 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>

On behalf of the Python development team, I'm gleeful to announce the second
alpha release of Python 2.7.

Python 2.7 is scheduled to be the last major version in the 2.x series.  It
includes many features that were first released in Python 3.1.  The faster io
module, the new nested with statement syntax, improved float repr, and the
memoryview object have been backported from 3.1. Other features include an
ordered dictionary implementation, unittests improvements, and support for ttk
Tile in Tkinter.  For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7 visit:

     http://www.python.org/download/releases/2.7/

Please note that this is a development release, intended as a preview of new
features for the community, and is thus not suitable for production use.

The 2.7 documentation can be found at:

     http://docs.python.org/2.7

Please consider trying Python 2.7 with your code and reporting any bugs you may
notice to:

     http://bugs.python.org


Have fun!

--
Benjamin Peterson
2.7 Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7's contributors)


From kmtracey at gmail.com  Sat Jan  9 19:48:12 2010
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 9 Jan 2010 13:48:12 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>

On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On behalf of the Python development team, I'm gleeful to announce the
> second
> alpha release of Python 2.7.
>
>
Well yay.  Django's test suite (1242 tests) runs with just one failure on
the 2.7 alpha 2 level, and that looks to be likely due to the improved
string/float rounding so not really a problem, just a difference.  That's
down from 104 failures and 40 errors with 2.7 alpha 1.

Note on the website page http://www.python.org/download/releases/2.7/ the
"Change log for this release" link is still pointing to the alpha 1
changelog.

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100109/d5c06f60/attachment-0003.htm>

From benjamin at python.org  Sat Jan  9 19:51:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 12:51:11 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <1afaf6161001091051kb1ba08q9b96094af16c12fa@mail.gmail.com>

2010/1/9 Karen Tracey <kmtracey at gmail.com>:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>
> Well yay.? Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.? That's
> down from 104 failures and 40 errors with 2.7 alpha 1.

Excellent!

>
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Thanks. I'll fix that.

>



-- 
Regards,
Benjamin


From skip at pobox.com  Sat Jan  9 21:00:44 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:00:44 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
Message-ID: <19272.57452.972234.643008@montanaro.dyndns.org>

How much of the Unladen Swallow cPickle speedups have been incorporated into
2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
two versions I currently care about at work - we will skip 2.5 entirely).
It appears some of their speedups may have already been merged to trunk, but
I'm not sure how much.  If a patch to merge this to 2.7 is already under
consideration I won't look at it, but I interpreted Collin Winter's response
to my query on the u-s mailing list that not everything has been done yet.

Thx,

Skip



From martin at v.loewis.de  Sat Jan  9 21:09:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 09 Jan 2010 21:09:27 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T160248-501@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
Message-ID: <4B48E277.9010408@v.loewis.de>

Antoine Pitrou wrote:
> Walter D?rwald <walter <at> livinglogic.de> writes:
>> On the surface this looks like there's an encoding named "BOM", but 
>> looking at your patch I found that the check is still done in 
>> TextIOWrapper. IMHO the best approach would to the implement a *real* 
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
>> the IO library. It could even be developed as a standalone project and 
>> published in the Cheeseshop.
> 
> Sorry but this is missing the point. The point here is to improve the open()
> function. I'm sure people who know about encodings are able to install the
> chardet library or even whip up their own BOM detection routine...

How does the requirement that it be implemented as a codec miss the
point?

FWIW, I agree with Walter that if it is provided through the encoding=
argument, it should be a codec. If it is built into the open function
(for whatever reason), it must be provided by some other parameter.

I do see the point that it becomes available to end users only when
released as part of Python. However, this *also* means that applications
won't be using it for another three years or so, since they'll have to
support older Python versions as well (unless it is integrated in the
case where no encoding is specified).

Regards,
Martin


From pjenvey at underboss.org  Sat Jan  9 21:09:29 2010
From: pjenvey at underboss.org (Philip Jenvey)
Date: Sat, 9 Jan 2010 12:09:29 -0800
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <19272.57452.972234.643008@montanaro.dyndns.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>


On Jan 9, 2010, at 12:00 PM, skip at pobox.com wrote:

> How much of the Unladen Swallow cPickle speedups have been incorporated into
> 2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
> two versions I currently care about at work - we will skip 2.5 entirely).
> It appears some of their speedups may have already been merged to trunk, but
> I'm not sure how much.  If a patch to merge this to 2.7 is already under
> consideration I won't look at it, but I interpreted Collin Winter's response
> to my query on the u-s mailing list that not everything has been done yet.

They've documented their upstream patches here:

http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

--
Philip Jenvey



From skip at pobox.com  Sat Jan  9 21:21:20 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:21:20 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
Message-ID: <19272.58688.173011.412712@montanaro.dyndns.org>


    Philip> They've documented their upstream patches here:

    Philip> http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

Thanks.  That will help immensely.

Skip



From solipsis at pitrou.net  Sat Jan  9 21:28:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:28:12 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
Message-ID: <loom.20100109T212555-508@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.




From solipsis at pitrou.net  Sat Jan  9 21:32:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:32:27 +0000 (UTC)
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <loom.20100109T213033-976@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> If a patch to merge this to 2.7 is already under
> consideration I won't look at it,

Why won't you look at it? :)
Actually, if these patches are to be merged someone should certainly look at
them, and do the (possibly) remaining work.

http://bugs.python.org/issue5683
http://bugs.python.org/issue5671

Thank you

Antoine.




From skip at pobox.com  Sat Jan  9 21:43:42 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:43:42 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
In-Reply-To: <loom.20100109T213033-976@post.gmane.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<loom.20100109T213033-976@post.gmane.org>
Message-ID: <19272.60030.25319.269597@montanaro.dyndns.org>

>>>>> "Antoine" == Antoine Pitrou <solipsis at pitrou.net> writes:

    Antoine> <skip <at> pobox.com> writes:
    >> 
    >> If a patch to merge this to 2.7 is already under
    >> consideration I won't look at it,

    Antoine> Why won't you look at it? :)

I meant I wouldn't look at developing one.

Skip


From regebro at gmail.com  Sat Jan  9 23:14:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 23:14:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>

On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Right. It seems like encoding=None is the right way to go there.
encoding='BOM' would probably only work if 'BOM' isn't an encoding but
a special tag, which is ugly.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Sun Jan 10 00:25:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 23:25:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
Message-ID: <4B49105E.303@voidspace.org.uk>

On 09/01/2010 22:14, Lennart Regebro wrote:
> On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>    
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
>>      
> Right. It seems like encoding=None is the right way to go there.
> encoding='BOM' would probably only work if 'BOM' isn't an encoding but
> a special tag, which is ugly.
>
>    
I would rather see it as the default behavior for open without an 
encoding specified.

I know Guido has expressed a preference against this so I won't continue 
to flog it.

The current behavior however is that we have a 'guessing' algorithm 
based on the platform default. Currently if you open a text file in read 
mode that has a UTF-8 signature, but the platform default is something 
other than UTF-8, then we open the file using what is likely to be the 
incorrect encoding. Looking for the signature seems to be better 
behaviour in that case.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Sun Jan 10 00:40:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Jan 2010 00:40:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <4B4913DF.7050801@v.loewis.de>

>> How does the requirement that it be implemented as a codec miss the
>> point?
> 
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Yes - however, Victor currently apparently *doesn't* want it to be the
default, but wants the user to specify encoding="BOM". If so, it isn't
the default, and it is easy to implement as a codec.

>> FWIW, I agree with Walter that if it is provided through the encoding=
>> argument, it should be a codec. If it is built into the open function
>> (for whatever reason), it must be provided by some other parameter.
> 
> Why not simply encoding=None?

I don't mind. Please re-read Walter's message - it only said that
*if* this is activated through encoding="BOM", *then* it must be
a codec, and could be on PyPI. I don't think Walter was talking about
the case "it is not activated through encoding='BOM'" *at all*.

> The default value should provide the most useful
> behaviour possible. Forcing users to choose between two different autodetection
> strategies (encoding=None and another one) is a little insane IMO.

That wouldn't disturb me much. There are a lot of things in that area
that are a little insane, starting with Microsoft Windows :-)

Regards,
Martin


From henning.vonbargen at arcor.de  Sun Jan 10 12:10:02 2010
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 10 Jan 2010 12:10:02 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <mailman.2987.1263079529.28904.python-dev@python.org>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
Message-ID: <4B49B58A.4040103@arcor.de>

If Python should support BOM when reading text files,
it should also be able to *write* such files.

An encoding="BOM" argument wouldn't help here, because
it does not specify which encoding to use actually:
UFT-8, UTF-16-LE or what?

That would be a point against encoding="BOM" and
pro an additional keyword argument "use_bom" or whatever
with the following values:

None: default (old) behaviour: don't handle BOM at all

True: reading: expect BOM (raising an exception if it's
                missing). The encoding argument must be None
                or it must match the encoding implied by the
                BOM
       writing: write a BOM. The encoding argument must be
                one of the UTF encodings.
False: reading: If a BOM is present, use it to determine the
                file encoding. The encoding argument must
                be None or it must match the encoding implied by
                the BOM. (*)
                Otherwise, use the encoding argument to determine
                the encoding.
        writing: do not write a BOM. Use the encoding argument.

(*) This is a question of taste. I think some people would prefer
     a fourth value "AUTO" instead, or to swap the behaviour of
     None and False.

Henning

P.S. To make things worse, I have sometimes seen XML files with a
UTF-8 BOM, but an XML encoding declaration of "iso-8859-1".
For such files, whatever you guess will be wrong anyway...


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From brett at python.org  Sun Jan 10 19:51:26 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 10:51:26 -0800
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
Message-ID: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>

Nick Coghlan thought I should forward this to python-dev so people are aware
of this change and why it occurred (plus it indirectly informs Guido I
finally finished the work).

---------- Forwarded message ----------
From: brett.cannon <python-checkins at python.org>
Date: Sat, Jan 9, 2010 at 18:56
Subject: [Python-checkins] r77402 - in python/trunk:
Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS Python/_warnings.c
To: python-checkins at python.org


Author: brett.cannon
Date: Sun Jan 10 03:56:19 2010
New Revision: 77402

Log:
DeprecationWarning is now silent by default.

This was originally suggested by Guido, discussed on the stdlib-sig mailing
list, and given the OK by Guido directly to me. What this change essentially
means is that Python has taken a policy of silencing warnings that are only
of interest to developers by default. This should prevent users from seeing
warnings which are triggered by an application being run against a new
interpreter before the app developer has a chance to update their code.

Closes issue #7319. Thanks to Antoine Pitrou, Ezio Melotti, and Brian Curtin
for helping with the issue.


Modified:
  python/trunk/Doc/library/warnings.rst
  python/trunk/Lib/test/test_ascii_formatd.py
  python/trunk/Lib/test/test_exceptions.py
  python/trunk/Lib/warnings.py
  python/trunk/Misc/NEWS
  python/trunk/Python/_warnings.c

Modified: python/trunk/Doc/library/warnings.rst
==============================================================================
--- python/trunk/Doc/library/warnings.rst       (original)
+++ python/trunk/Doc/library/warnings.rst       Sun Jan 10 03:56:19 2010
@@ -59,7 +59,7 @@
 | :exc:`UserWarning`               | The default category for :func:`warn`.
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`DeprecationWarning`        | Base category for warnings about
deprecated   |
-|                                  | features.
        |
+|                                  | features (ignored by default).
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`SyntaxWarning`             | Base category for warnings about
dubious      |
 |                                  | syntactic features.
        |
@@ -89,6 +89,9 @@
 standard warning categories.  A warning category must always be a subclass
of
 the :exc:`Warning` class.

+.. versionchanged:: 2.7
+   :exc:`DeprecationWarning` is ignored by default.
+

 .. _warning-filter:

@@ -148,14 +151,6 @@
 :mod:`warnings` module parses these when it is first imported (invalid
options
 are ignored, after printing a message to ``sys.stderr``).

-The warnings that are ignored by default may be enabled by passing
:option:`-Wd`
-to the interpreter. This enables default handling for all warnings,
including
-those that are normally ignored by default. This is particular useful for
-enabling ImportWarning when debugging problems importing a developed
package.
-ImportWarning can also be enabled explicitly in Python code using::
-
-   warnings.simplefilter('default', ImportWarning)
-

 .. _warning-suppress:

@@ -226,6 +221,37 @@
 entries from the warnings list before each new operation).


+Updating Code For New Versions of Python
+----------------------------------------
+
+Warnings that are only of interest to the developer are ignored by default.
As
+such you should make sure to test your code with typically ignored warnings
+made visible. You can do this from the command-line by passing
:option:`-Wd`
+to the interpreter (this is shorthand for :option:`-W default`).  This
enables
+default handling for all warnings, including those that are ignored by
default.
+To change what action is taken for encountered warnings you simply change
what
+argument is passed to :option:`-W`, e.g. :option:`-W error`. See the
+:option:`-W` flag for more details on what is possible.
+
+To programmatically do the same as :option:`-Wd`, use::
+
+  warnings.simplefilter('default')
+
+Make sure to execute this code as soon as possible. This prevents the
+registering of what warnings have been raised from unexpectedly influencing
how
+future warnings are treated.
+
+Having certain warnings ignored by default is done to prevent a user from
+seeing warnings that are only of interest to the developer. As you do not
+necessarily have control over what interpreter a user uses to run their
code,
+it is possible that a new version of Python will be released between your
+release cycles.  The new interpreter release could trigger new warnings in
your
+code that were not there in an older interpreter, e.g.
+:exc:`DeprecationWarning` for a module that you are using. While you as a
+developer want to be notified that your code is using a deprecated module,
to a
+user this information is essentially noise and provides no benefit to them.
+
+
 .. _warning-functions:

 Available Functions

Modified: python/trunk/Lib/test/test_ascii_formatd.py
==============================================================================
--- python/trunk/Lib/test/test_ascii_formatd.py (original)
+++ python/trunk/Lib/test/test_ascii_formatd.py Sun Jan 10 03:56:19 2010
@@ -4,6 +4,7 @@

 import unittest
 from test_support import check_warnings, run_unittest, cpython_only
+import warnings


 class FormatDeprecationTests(unittest.TestCase):
@@ -17,6 +18,7 @@
        buf = create_string_buffer(' ' * 100)

        with check_warnings() as w:
+            warnings.simplefilter('default')
            PyOS_ascii_formatd(byref(buf), sizeof(buf), '%+.10f',
                               c_double(10.0))
            self.assertEqual(buf.value, '+10.0000000000')

Modified: python/trunk/Lib/test/test_exceptions.py
==============================================================================
--- python/trunk/Lib/test/test_exceptions.py    (original)
+++ python/trunk/Lib/test/test_exceptions.py    Sun Jan 10 03:56:19 2010
@@ -309,6 +309,7 @@
        # BaseException.__init__ triggers a deprecation warning.
        exc = BaseException("foo")
        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('default')
            self.assertEquals(exc.message, "foo")
        self.assertEquals(len(w), 1)
        self.assertEquals(w[0].category, DeprecationWarning)

Modified: python/trunk/Lib/warnings.py
==============================================================================
--- python/trunk/Lib/warnings.py        (original)
+++ python/trunk/Lib/warnings.py        Sun Jan 10 03:56:19 2010
@@ -383,8 +383,8 @@
 # Module initialization
 _processoptions(sys.warnoptions)
 if not _warnings_defaults:
-    simplefilter("ignore", category=PendingDeprecationWarning, append=1)
-    simplefilter("ignore", category=ImportWarning, append=1)
+    for cls in (DeprecationWarning, PendingDeprecationWarning,
ImportWarning):
+        simplefilter("ignore", category=cls, append=True)
    bytes_warning = sys.flags.bytes_warning
    if bytes_warning > 1:
        bytes_action = "error"

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sun Jan 10 03:56:19 2010
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------

+- Issue #7319: Silence DeprecationWarning by default.
+
 - Issue #2335: Backport set literals syntax from Python 3.x.

 Library

Modified: python/trunk/Python/_warnings.c
==============================================================================
--- python/trunk/Python/_warnings.c     (original)
+++ python/trunk/Python/_warnings.c     Sun Jan 10 03:56:19 2010
@@ -85,10 +85,10 @@

    default_action = get_warnings_attr("defaultaction");
    if (default_action == NULL) {
-       if (PyErr_Occurred()) {
-           return NULL;
-       }
-       return _default_action;
+        if (PyErr_Occurred()) {
+            return NULL;
+        }
+        return _default_action;
    }

    Py_DECREF(_default_action);
@@ -202,12 +202,12 @@

    mod_str = PyString_AsString(filename);
    if (mod_str == NULL)
-           return NULL;
+        return NULL;
    len = PyString_Size(filename);
    if (len < 0)
        return NULL;
    if (len >= 3 &&
-       strncmp(mod_str + (len - 3), ".py", 3) == 0) {
+            strncmp(mod_str + (len - 3), ".py", 3) == 0) {
        module = PyString_FromStringAndSize(mod_str, len-3);
    }
    else {
@@ -251,7 +251,7 @@

    name = PyObject_GetAttrString(category, "__name__");
    if (name == NULL)  /* XXX Can an object lack a '__name__' attribute? */
-           return;
+        return;

    f_stderr = PySys_GetObject("stderr");
    if (f_stderr == NULL) {
@@ -341,7 +341,7 @@
        rc = already_warned(registry, key, 0);
        if (rc == -1)
            goto cleanup;
-       else if (rc == 1)
+        else if (rc == 1)
            goto return_none;
        /* Else this warning hasn't been generated before. */
    }
@@ -492,9 +492,9 @@
    /* Setup filename. */
    *filename = PyDict_GetItemString(globals, "__file__");
    if (*filename != NULL) {
-           Py_ssize_t len = PyString_Size(*filename);
+            Py_ssize_t len = PyString_Size(*filename);
        const char *file_str = PyString_AsString(*filename);
-           if (file_str == NULL || (len < 0 && PyErr_Occurred()))
+            if (file_str == NULL || (len < 0 && PyErr_Occurred()))
            goto handle_error;

        /* if filename.lower().endswith((".pyc", ".pyo")): */
@@ -506,10 +506,10 @@
                tolower(file_str[len-1]) == 'o'))
        {
            *filename = PyString_FromStringAndSize(file_str, len-1);
-               if (*filename == NULL)
-                       goto handle_error;
-           }
-           else
+            if (*filename == NULL)
+                goto handle_error;
+        }
+        else
            Py_INCREF(*filename);
    }
    else {
@@ -536,8 +536,8 @@
            else {
                /* embedded interpreters don't have sys.argv, see bug
#839151 */
                *filename = PyString_FromString("__main__");
-                   if (*filename == NULL)
-                       goto handle_error;
+                if (*filename == NULL)
+                    goto handle_error;
            }
        }
        if (*filename == NULL) {
@@ -839,26 +839,29 @@
 static PyObject *
 init_filters(void)
 {
-    PyObject *filters = PyList_New(3);
+    PyObject *filters = PyList_New(4);
    const char *bytes_action;
    if (filters == NULL)
        return NULL;

    PyList_SET_ITEM(filters, 0,
+                    create_filter(PyExc_DeprecationWarning, "ignore"));
+    PyList_SET_ITEM(filters, 1,
                    create_filter(PyExc_PendingDeprecationWarning,
"ignore"));
-    PyList_SET_ITEM(filters, 1, create_filter(PyExc_ImportWarning,
"ignore"));
+    PyList_SET_ITEM(filters, 2, create_filter(PyExc_ImportWarning,
"ignore"));
    if (Py_BytesWarningFlag > 1)
        bytes_action = "error";
    else if (Py_BytesWarningFlag)
        bytes_action = "default";
    else
        bytes_action = "ignore";
-    PyList_SET_ITEM(filters, 2, create_filter(PyExc_BytesWarning,
+    PyList_SET_ITEM(filters, 3, create_filter(PyExc_BytesWarning,
                    bytes_action));

    if (PyList_GET_ITEM(filters, 0) == NULL ||
        PyList_GET_ITEM(filters, 1) == NULL ||
-        PyList_GET_ITEM(filters, 2) == NULL) {
+        PyList_GET_ITEM(filters, 2) == NULL ||
+        PyList_GET_ITEM(filters, 3) == NULL) {
        Py_DECREF(filters);
        return NULL;
     }
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db6da5fd/attachment-0003.htm>

From victor.stinner at haypocalc.com  Sun Jan 10 20:22:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 10 Jan 2010 20:22:10 +0100
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
Message-ID: <201001102022.10259.victor.stinner@haypocalc.com>

Hi,

Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
> Nick Coghlan thought I should forward this to python-dev so people are
>  aware of this change and why it occurred (plus it indirectly informs Guido
>  I finally finished the work).

Thanks to copy this mail to the python-dev mailing list.

What is the recommanded way to get the previous behaviour (print deprecation 
warnings once)? Is there a command line option? Is it "-W default"?

-- 
Victor Stinner
http://www.haypocalc.com/


From benjamin at python.org  Sun Jan 10 20:23:54 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 13:23:54 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <201001102022.10259.victor.stinner@haypocalc.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
	<201001102022.10259.victor.stinner@haypocalc.com>
Message-ID: <1afaf6161001101123g366d80dbjeaa80f1a632605e2@mail.gmail.com>

2010/1/10 Victor Stinner <victor.stinner at haypocalc.com>:
> Hi,
>
> Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
>> Nick Coghlan thought I should forward this to python-dev so people are
>> ?aware of this change and why it occurred (plus it indirectly informs Guido
>> ?I finally finished the work).
>
> Thanks to copy this mail to the python-dev mailing list.
>
> What is the recommanded way to get the previous behaviour (print deprecation
> warnings once)? Is there a command line option? Is it "-W default"?

That commit conveniently adds documentation answering that question. :)



-- 
Regards,
Benjamin


From nas at arctrix.com  Sun Jan 10 20:30:09 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:30:09 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <hid9s1$i05$1@ger.gmane.org>

Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm gleeful to announce
> the second alpha release of Python 2.7.

Thanks to everyone who contributed.

> Python 2.7 is scheduled to be the last major version in the 2.x
> series.

Has this really been decided already? Maybe I missed it.  In my
opinion, it does Python's reputation harm to make such a statement.
Conservative users (probably the vast majority of Python users)
don't like to hear that software they are considering using is
nearing the end of its life.  What does it gain us to announce that
the 2.x branch is dead aside from bugfixes?

I propose that the 2.x branch be treated like 2.x.y branches: as
long as there is sufficient volunteer labour, it should continue to
live.  In order to avoid wasted development effort, it would be
prudent to announce that unless a 2.8 release manager steps up,
whatever is committed to the 2.x branch after the 2.7 release may
never get released.

Said another way, it's okay for the Python developers to decide to
abandon 2.x and put their efforts into 3.x. It's not okay for them
to prevent others from continuing to work on 2.x or to somehow make
2.x look worse so 3.x looks better. Python 3 needs to stand on its
own terms and I'm confident it can.

Best regards,

  Neil



From brett at python.org  Sun Jan 10 21:09:08 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:09:08 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>

On Sun, Jan 10, 2010 at 11:30, Neil Schemenauer <nas at arctrix.com> wrote:

> Benjamin Peterson <benjamin at python.org> wrote:
> > On behalf of the Python development team, I'm gleeful to announce
> > the second alpha release of Python 2.7.
>
> Thanks to everyone who contributed.
>
> > Python 2.7 is scheduled to be the last major version in the 2.x
> > series.
>
> Has this really been decided already? Maybe I missed it.


More or less. It was first discussed at the language summit last year and
has come up here a couple of times. If needed we can make it official in
terms of lifetime of 2.7, etc. at the language summit this year.


>  In my
> opinion, it does Python's reputation harm to make such a statement.
> Conservative users (probably the vast majority of Python users)
> don't like to hear that software they are considering using is
> nearing the end of its life.  What does it gain us to announce that
> the 2.x branch is dead aside from bugfixes?
>
> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.
>
> Said another way, it's okay for the Python developers to decide to
> abandon 2.x and put their efforts into 3.x. It's not okay for them
> to prevent others from continuing to work on 2.x or to somehow make
> 2.x look worse so 3.x looks better. Python 3 needs to stand on its
> own terms and I'm confident it can.
>

I don't think ending the 2.x series at 2.7 makes it look bad compared to
3.2; it's simply the end of a development line like any other software
project. I suspect 2.7 will have a protracted bugfix window because so much
code runs on 2.x exclusively at the moment. And if core developers want to
continue to backport fixes past two years  from release they can (or however
long we decide to officially support 2.7).

No one is saying people still can't work on the code, just that python-dev
as an entity is not going to focus its effort on the 2.x series anymore and
people should not rely upon us to continue to focus new development efforts
in that series. If there are core developers who want to continue to do
bugfix releases then that's fine and I am happy to flag patches as needing
backports and let other's do the work after the standard two year
maintenance cycle, but I know I do not want to be held accountable as a core
developer to keep the 2.x going indefinitely. Maintaining four branches is
enough of a reason in my book to not keep the 2.x series going.

If there really is an outcry on this we can re-visit the issue, but as of
right now we need to move forward at some point and 2.7 seems like that good
point.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/4bff0434/attachment-0003.htm>

From ncoghlan at gmail.com  Sun Jan 10 21:23:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 11 Jan 2010 06:23:27 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A373F.9050601@gmail.com>

Neil Schemenauer wrote:
> In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

The announcement is precisely to avoid the situation where people commit
new features to the 2.x main line of development (after the 2.7
maintenance branch is created) in the expectation that they will be
released as part of a hypothetical 2.8 release.

Whether that info needs to be in each and every 2.7 announcement...
probably not. It isn't really info for users of Python, just for
developers of Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Sun Jan 10 21:25:29 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:25:29 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
Message-ID: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>

Michael has given me the hg transition/stdlib time slot at the language
summit this year. In regards to that I plan to lead a discussion on:

* where we are at w/ the Hg transition (Dirkjan should be there and I did a
blog post on this topic recently:
http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
* argparse (PEP 389)
* brief mention on still wanting to break out the stdlib from CPython
* an official policy on extension modules? (i.e. must have a pure Python
implementation which can mean a ctypes implementation unless given an
explicit waiver)

That's everything from a stdlib perspective. I have half-baked ideas, but
nothing concrete enough to bring up unless people really want to discuss
stuff like how to potentially standardize module deprecation warnings, etc.

In terms of me personally, I do plan to bring up at some point during the
summit these points which don't squarely fit during my time slot:

* an official unofficial policy on how new proposed features should come to
us (i.e. working code to python-ideas with a shell of a PEP that includes
abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
* any changes needed to the issue tracker to help with the workflow? (stage
field seems like a failed experiment and we now have several effective
triage people who can help w/ guiding changes)

If there is something missing from the stdlib discussion that you think
should be brought up at the summit let me know. And if there is something
here you want to discuss before the summit let me know and I can start a
separate thread on it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/f16584e2/attachment-0003.htm>

From dirkjan at ochtman.nl  Sun Jan 10 22:52:00 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 10 Jan 2010 22:52:00 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:25, Brett Cannon <brett at python.org> wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)

Unfortunately my flight doesn't land until about the time the
Mercurial slot ends, so while I'll be there later on that afternoon
for sprinting or questions or anything, I won't be at the actual
Mercurial talk at the summit.

Cheers,

Dirkjan


From fuzzyman at voidspace.org.uk  Sun Jan 10 23:27:58 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 10 Jan 2010 22:27:58 +0000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
Message-ID: <4B4A546E.8010808@voidspace.org.uk>

On 10/01/2010 21:52, Dirkjan Ochtman wrote:
> On Sun, Jan 10, 2010 at 21:25, Brett Cannon<brett at python.org>  wrote:
>    
>> Michael has given me the hg transition/stdlib time slot at the language
>> summit this year. In regards to that I plan to lead a discussion on:
>> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
>> blog post on this topic recently:
>> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
>>      
> Unfortunately my flight doesn't land until about the time the
> Mercurial slot ends, so while I'll be there later on that afternoon
> for sprinting or questions or anything, I won't be at the actual
> Mercurial talk at the summit.
>
>    
We will probably be in 'open discussion' when you arrive so it will 
still be good to hear from you on the topic and there maybe questions 
that can't be answered until you arrive... :-)

Michael

> Cheers,
>
> Dirkjan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Mon Jan 11 00:02:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:02:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A5C69.7090007@v.loewis.de>

> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

I think it's more difficult than that. It also takes developer effort
to *commit* to the trunk. So if the trunk continued to live, would
it still be ok if I closed a 2.x feature request as "won't fix", or
only committed the new feature to the 3.x branch?

As for old branches: they *don't* live in the way you claim (i.e.
remain open with changes potentially just not released). Instead,
at some point, they are frozen to bug-fix only, then to security-fix
only, and then to no change at all.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 00:07:58 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:07:58 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A373F.9050601@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>
Message-ID: <4B4A5DCE.3070909@v.loewis.de>

> The announcement is precisely to avoid the situation where people commit
> new features to the 2.x main line of development (after the 2.7
> maintenance branch is created) in the expectation that they will be
> released as part of a hypothetical 2.8 release.
> 
> Whether that info needs to be in each and every 2.7 announcement...
> probably not. It isn't really info for users of Python, just for
> developers of Python.

I think the announcement is indeed also for the users, and I would
prefer it to stay in the release announcements, so that people will
know for sure (and speak up) before the 2.7 release happens.

As for decisions: I don't think there was an official BDFL pronouncent,
but I recall Guido posting a message close to that, proposing that 2.7
will be a release that will see bug fix releases for an indefinite
period of time (where indefinite != infinite). This was shortly after
him proposing that perhaps we shouldn't make a 2.7 release at all, and
stop at 2.6.

As for such a decision giving a bad light on Python: I don't think that
will be the case. Instead, I hear many users surprised for how long
we have been maintaining to parallel versions - "that must have taken
a lot of effort". So everybody will likely understand that enough is
enough.

Regards,
Martin


From benjamin at python.org  Mon Jan 11 01:07:35 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 18:07:35 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>

Consider this program:

class Descr(object):
    def __init__(self, name):
        self.name = name
    def __set__(self, instance, what):
        instance.__dict__[self.name] = what

class X(object):
    attr = Descr("attr")

x = X()
print(x.attr)
x.attr = 42
print(x.attr)

It gives in output:

<__main__.Descr object at 0x7fe1c9b28150>
42

The documentation [1] says that Descr is a data descriptor because it
defines the __set__ method. It also states that data descriptors
always override the value in the instance dictionary. So, the second
line should also be the descriptor object according to the
documentation.

My question is: Is this a doc bug or a implementation bug? If the
former, it will be the description of a data descriptor much less
consistent, since it will require that a __get__ method be present,
too. If the latter, the fix may break some programs relying on the
ability to "cache" a value in the instance dictionary.

[1] http://docs.python.org/reference/datamodel#invoking-descriptors


-- 
Regards,
Benjamin


From amauryfa at gmail.com  Mon Jan 11 01:51:09 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 11 Jan 2010 01:51:09 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>

Hi,

2010/1/11 Benjamin Peterson <benjamin at python.org>:
> Consider this program:
>
> class Descr(object):
> ? ?def __init__(self, name):
> ? ? ? ?self.name = name
> ? ?def __set__(self, instance, what):
> ? ? ? ?instance.__dict__[self.name] = what
>
> class X(object):
> ? ?attr = Descr("attr")
>
> x = X()
> print(x.attr)
> x.attr = 42
> print(x.attr)
>
> It gives in output:
>
> <__main__.Descr object at 0x7fe1c9b28150>
> 42
>
> The documentation [1] says that Descr is a data descriptor because it
> defines the __set__ method. It also states that data descriptors
> always override the value in the instance dictionary. So, the second
> line should also be the descriptor object according to the
> documentation.
>
> My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
>
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

Quoting the documentation:
"""Normally, data descriptors define both __get__() and __set__(),
while non-data descriptors have just the __get__() method.
"""
Your example is neither a data descriptor nor a non-data descriptor...

The thing that worries me a bit is the "x.attr" returning the Descr
object. Descriptors should remain at the class level, and instance
should only see values. I'd prefer an AttributeError in this case.

-- 
Amaury Forgeot d'Arc


From benjamin at python.org  Mon Jan 11 02:00:32 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 19:00:32 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
Message-ID: <1afaf6161001101700u21006708l2ef62bfa257e4ce7@mail.gmail.com>

2010/1/10 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...

See the footnote: http://docs.python.org/reference/datamodel#id7

>
> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.

Far too late for that, I'm afraid.



-- 
Regards,
Benjamin


From nas at arctrix.com  Mon Jan 11 02:44:57 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:44:57 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
Message-ID: <20100111014457.GA5289@arctrix.com>

On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> I don't think ending the 2.x series at 2.7 makes it look bad
> compared to 3.2; it's simply the end of a development line like
> any other software project. I suspect 2.7 will have a protracted
> bugfix window because so much code runs on 2.x exclusively at the
> moment.

I would guess over 99% of all Python code written doesn't run on
Python 3.  Given that, I think it is premature to close the door on
new major versions of Python 2.x. Also, we as a project should be
careful not to present the image that Python 2.x will not be
supported in the future.

> If there really is an outcry on this we can re-visit the issue,
> but as of right now we need to move forward at some point and 2.7
> seems like that good point.

I think that's bad PR.  If I had a successful product, I would not
announce its end of life just to see how many customers scream and
then decide if I should devote more resources to continue
maintaining it.

IMHO, the release notes should say something like:

     After the Python 2.7 release, the focus of Python development
     will be on Python 3.  There will continue to be maintainance
     releases of Python 2.x.


trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil


From tjreedy at udel.edu  Mon Jan 11 03:26:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Jan 2010 21:26:34 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <hie28p$joo$1@ger.gmane.org>

On 1/10/2010 8:44 PM, Neil Schemenauer wrote:
> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
>> I don't think ending the 2.x series at 2.7 makes it look bad
>> compared to 3.2; it's simply the end of a development line like
>> any other software project. I suspect 2.7 will have a protracted
>> bugfix window because so much code runs on 2.x exclusively at the
>> moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.

If the removal of old features had been done in the 2.x series, as once 
planned (Guido originally proposed removing the old meaning of int / int 
in 2.5) the same more or less would be true of 2.7. It is past time for 
other old and now duplicated features to be removed also.

   Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>
>> If there really is an outcry on this we can re-visit the issue,
>> but as of right now we need to move forward at some point and 2.7
>> seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.

Python is not being ended, but upgraded (with bloating duplications 
removed). Think of 3.1 as 2.8 with a new name.

tjr



From lrekucki at gmail.com  Mon Jan 11 04:26:48 2010
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Mon, 11 Jan 2010 04:26:48 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <dfbefb091001101926l366043f8mb95f196ba14f9c9@mail.gmail.com>

> Date: Mon, 11 Jan 2010 01:51:09 +0100
> From: "Amaury Forgeot d'Arc" <amauryfa at gmail.com>
> To: Benjamin Peterson <benjamin at python.org>
> Cc: Python Dev <python-dev at python.org>
> Subject: Re: [Python-Dev] Data descriptor doc/implementation
> ? ? ? ?inconsistency
> Message-ID:
> ? ? ? ?<e27efe131001101651y68e1da25je2a8d02f5c62ef19 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> 2010/1/11 Benjamin Peterson <benjamin at python.org>:
>> Consider this program:
>>
>> class Descr(object):
>> ? ?def __init__(self, name):
>> ? ? ? ?self.name = name
>> ? ?def __set__(self, instance, what):
>> ? ? ? ?instance.__dict__[self.name] = what
>>
>> class X(object):
>> ? ?attr = Descr("attr")
>>
>> x = X()
>> print(x.attr)
>> x.attr = 42
>> print(x.attr)
>>
>> It gives in output:
>>
>> <__main__.Descr object at 0x7fe1c9b28150>
>> 42
>>
>> The documentation [1] says that Descr is a data descriptor because it
>> defines the __set__ method. It also states that data descriptors
>> always override the value in the instance dictionary. So, the second
>> line should also be the descriptor object according to the
>> documentation.
>>
>> My question is: Is this a doc bug or a implementation bug? If the
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...
Actually, there is this footnote [1]:

""" A descriptor can define any combination of __get__(), __set__()
and __delete__(). If it does not define __get__(), then accessing the
attribute even on an instance will return the descriptor object
itself. If the descriptor defines __set__() and/or __delete__(), it is
a data descriptor; if it defines neither, it is a non-data descriptor.
"""

Which would mean Descr is actually a data descriptor without a
__get__(), so x.attr should always return the descriptor object itself
(at least in the docs).

> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.
>
> --
> Amaury Forgeot d'Arc

[1] http://docs.python.org/reference/datamodel#id7
-- 
Lukasz Rekucki


From brett at python.org  Mon Jan 11 04:46:04 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 19:46:04 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
Message-ID: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>

On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> > I don't think ending the 2.x series at 2.7 makes it look bad
> > compared to 3.2; it's simply the end of a development line like
> > any other software project. I suspect 2.7 will have a protracted
> > bugfix window because so much code runs on 2.x exclusively at the
> > moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x.


Well yeah; Python 3.0 is just over a year old with 3.1 -- the first robust
3.x release - is about six months old. From the beginning uptake was
expected to take years, not months, so saying that 3.x is not popular enough
seems premature.


> Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>

No one has said bugfixes will cease.


>
> > If there really is an outcry on this we can re-visit the issue,
> > but as of right now we need to move forward at some point and 2.7
> > seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.
>
>
I never said that I wanted to make this announcement specifically to provoke
outcries. I said if there happened to be outcries we could possibly visit
the issue again. Basically I was being considerate and trying to leave the
door open to discuss things in the future even though I don't see the
situation changing.


> IMHO, the release notes should say something like:
>
>     After the Python 2.7 release, the focus of Python development
>     will be on Python 3.  There will continue to be maintainance
>     releases of Python 2.x.
>

No because that suggests new features will be coming to 2.x which is not
going to happen. If you want to say there will be continual bugfix releases
for 2.7 as is par the course for Python and that the number of bugfix
releases might be more than normal then I am okay with that.


>
>
> trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil
>

That came and went already a couple months ago when we discussed stopping at
2.6 instead of 2.7 (see the various threads at
http://mail.python.org/pipermail/python-dev/2009-November/thread.html).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db586677/attachment-0003.htm>

From nas at arctrix.com  Mon Jan 11 05:05:07 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:05:07 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
Message-ID: <20100111040507.GA7200@arctrix.com>

On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> >     After the Python 2.7 release, the focus of Python development
> >     will be on Python 3.  There will continue to be maintainance
> >     releases of Python 2.x.
> 
> No because that suggests new features will be coming to 2.x which is not
> going to happen. If you want to say there will be continual bugfix releases
> for 2.7 as is par the course for Python and that the number of bugfix
> releases might be more than normal then I am okay with that.

Are you are saying that if someone steps up to merge the Unladen
Swallow features into a 2.8 release and someone also steps up to cut
the release that they will be prevented from doing so?

Also, are you presuming to channel the BDFL or was this dicussed
somewhere other than python-dev? Maybe I'm overreacting but I get
the feeling that the larger and less active segment of the Python
develpment team hasn't been involved in these decisions.

Best regards,

  Neil


From nas at arctrix.com  Mon Jan 11 05:17:54 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:17:54 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5C69.7090007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
Message-ID: <20100111041754.GB7200@arctrix.com>

On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
> [...] would it still be ok if I closed a 2.x feature request as
> "won't fix", or only committed the new feature to the 3.x branch?

Yes.  Non-bugfix development on 2.x would optional (i.e. done by
people who want to spend the time). Since language changes are now
out (an initiative I completely support), the majority of non-bugfix
changes would be optimizations and platform porting.

Regards,

  Neil


From brett at python.org  Mon Jan 11 06:06:15 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 21:06:15 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111040507.GA7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
Message-ID: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>

On Sun, Jan 10, 2010 at 20:05, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> > On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> > >     After the Python 2.7 release, the focus of Python development
> > >     will be on Python 3.  There will continue to be maintainance
> > >     releases of Python 2.x.
> >
> > No because that suggests new features will be coming to 2.x which is not
> > going to happen. If you want to say there will be continual bugfix
> releases
> > for 2.7 as is par the course for Python and that the number of bugfix
> > releases might be more than normal then I am okay with that.
>
> Are you are saying that if someone steps up to merge the Unladen
> Swallow features into a 2.8 release and someone also steps up to cut
> the release that they will be prevented from doing so?
>
>
I honestly don't know, but it's a possibility just like any other new
feature request. If people start taking the carrots we have added to 3.x and
backporting them to keep the 2.x series alive you are essentially making the
3.x DOA by negating its benefits which I personally don't agree with.


> Also, are you presuming to channel the BDFL or was this dicussed
> somewhere other than python-dev?


This was discussed; see the November 2009 threads. Guido actually suggested
ending the 2.x branch at 2.6 until people spoke up about the amount of stuff
already backported to 2.7 from 3.x because it was being assumed to be the
end of the series to warrant keeping it to help transition to 2.7.


> Maybe I'm overreacting but I get
> the feeling that the larger and less active segment of the Python
> develpment team hasn't been involved in these decisions.
>

This all came up in November from the 3rd through the 6th (four days) over a
ton of email traffic. This was not a snap decision but a heated discussion
where even Guido participated that culminated with the decision to stop at
2.7 for the 2.x series; this was not a smoked-filled room decision. I'm
sorry if people missed it when they weren't looking, but python-dev is
supposed to be the place to watch for this kind of stuff. I don't know how
else to bring this stuff to people's attention short of also on
python-committers, but developers are basically expected to be on both lists
so I don't see where anyone did anything wrong in terms of informing
developers.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/18c3bcd7/attachment-0003.htm>

From nas at arctrix.com  Mon Jan 11 06:27:13 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 23:27:13 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
Message-ID: <20100111052713.GA8211@arctrix.com>

On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> If people start taking the carrots we have added to 3.x and
> backporting them to keep the 2.x series alive you are essentially
> making the 3.x DOA by negating its benefits which I personally
> don't agree with.

I think we have got to the heart of our disagreement. Assume that
some superhuman takes all the backwards compatible goodies from 3.x
and merges them into 2.x. If that really would kill off Python 3
then I would proclaim it a project that deserved to die. Why should
the backwards incompatible changes be endured if they cannot justify
their existance by the benefit they provide?

I guess I have more confidence in Python 3 than you do. I don't see
why Python 2.x needs to be artificially limited so that Python 3 can
benefit.

Regards,

  Neil


From brett at python.org  Mon Jan 11 07:30:49 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 22:30:49 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
Message-ID: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:27, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> > If people start taking the carrots we have added to 3.x and
> > backporting them to keep the 2.x series alive you are essentially
> > making the 3.x DOA by negating its benefits which I personally
> > don't agree with.
>
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x. If that really would kill off Python 3
> then I would proclaim it a project that deserved to die. Why should
> the backwards incompatible changes be endured if they cannot justify
> their existance by the benefit they provide?
>
>
When I said "carrots" I meant everything that makes Python 3 the best
version of Python, including the backward-incompatible changes.

I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.
>

I don't view it as artificial but simply a focusing of the development team
on what has been determined as the future of Python by Guido and letting go
of the past.

IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev
saying "Python 3 is the future, but we are keeping the old Python 2.x around
because we don't have *that* much faith in the future we have laid out".
That's poison to Python 3 by showing a lack of confidence in the direction
that the BDFL and python-dev as a group has chosen. Now I could be wrong and
there could actually be a large number of active contributors who want to
keep the 2.x series going, but based on the discussion that occurred the
last time this came up I believe the guys who are keeping Python running are
ready to move on.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/36448e5f/attachment-0003.htm>

From regebro at gmail.com  Mon Jan 11 08:08:14 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 08:08:14 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <319e029f1001102308p5de87cber2131f3aec1abc9f0@mail.gmail.com>

On Mon, Jan 11, 2010 at 06:27, Neil Schemenauer <nas at arctrix.com> wrote:
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x.

The superhumans of core developers pretty much already have. That's
pretty much 2.7 you are talking about. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From chrism at plope.com  Mon Jan 11 08:27:03 2010
From: chrism at plope.com (Chris McDonough)
Date: Mon, 11 Jan 2010 02:27:03 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
Message-ID: <4B4AD2C7.1050703@plope.com>

Brett Cannon wrote:
> IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev 
> saying "Python 3 is the future, but we are keeping the old Python 2.x 
> around because we don't have *that* much faith in the future we have 
> laid out". That's poison to Python 3 by showing a lack of confidence in 
> the direction that the BDFL and python-dev as a group has chosen. Now I 
> could be wrong and there could actually be a large number of active 
> contributors who want to keep the 2.x series going, but based on the 
> discussion that occurred the last time this came up I believe the guys 
> who are keeping Python running are ready to move on.

I think this is mostly just a branding issue.  Once the folks who have 
historically kept Python 2 running really *do* move on, there will be other 
folks who will step up and become maintainers out of necessity: those stuck on 
old platforms, permanent stragglers, people who for whatever reason actually 
like Python 2 better, etc.  It's just going to happen, there's really nothing 
anybody can do about it.

I don't think there's anything wrong with this.  If such a group of folks wants 
to get together and create another Python 2.X release, there's nothing stopping 
them from doing so except the above branding declaration.  I'd personally not 
close the door on allowing these folks to call such an effort "Python 2".

- C



From martin at v.loewis.de  Mon Jan 11 09:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:06:16 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
	<20100111041754.GB7200@arctrix.com>
Message-ID: <4B4ADBF8.3030803@v.loewis.de>

Neil Schemenauer wrote:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time). 

I think *that* would give a very bad impression of Python. Depending
whom you deal with, the new feature you want may or may not get
added to the code base. Contributors would feel even more stranded
than they do now, where it may take several years to get a patch
reviewed, as you then could submit a patch, and pray that a comitter
reviews it who believes in future 2.x releases.

The point of setting policies is that it gives every user (contributors,
committers, and "end-user" developers) a reliable foundation for
expectations.

Regards,
Martin


From arcriley at gmail.com  Mon Jan 11 09:06:10 2010
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 11 Jan 2010 03:06:10 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AD2C7.1050703@plope.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com> 
	<4B4AD2C7.1050703@plope.com>
Message-ID: <bad82a81001110006n3cacd953g98fb25e96330ee89@mail.gmail.com>

after all these years, some people still run AmigaOS.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/4fdcbb4e/attachment-0003.htm>

From martin at v.loewis.de  Mon Jan 11 09:11:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:11:25 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <4B4ADD2D.6070906@v.loewis.de>

> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.

Why that? It is a fact: 2.x will not be supported, in some future.
Should we lie to users?

>      After the Python 2.7 release, the focus of Python development
>      will be on Python 3.  There will continue to be maintainance
>      releases of Python 2.x.

It shouldn't say that, because it wouldn't be true.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 09:18:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:18:30 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <4B4ADED6.5080207@v.loewis.de>

> I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.

Because it takes too much time. Too much of my time, but apparently
also too much of other people's time.

Of course, the less active fraction of Python contributors may not
notice, since they just chose to not contribute (which, of course,
is fine). However, asking me to work twice as much as I want to
on the project to keep two branches alive is just unfair.

This has nothing to do with pushing 3.x, but all with managing
available manpower and still providing quality software.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 09:42:51 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 09:42:51 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>

This is what it says now:

"Python 2.7 is scheduled to be the last major version in the 2.x
series before it moves into 5 years of bugfix-only mode. "

And during this discussion it has been noted that others than the core
python team might pick up Python 2 and make releases. So maybe we can
and this discussion by changing that line in future releases to be:

"Python 2.7 is scheduled to be the last major version in the 2.x
series released by the Python Software Foundation before it moves into
5 years of bugfix-only mode. "

That doesn't exclude *others* making a Python 2.8 that could include
all sorts of crazy features. :)

This is mainly just to get the pointless discussion over with. The
current Python core team don't want to make a 2.8, so that will not
happen. Someone else might, but as Chris points out, they won't step
up until they have to, which is probably at least two years from now.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 10:06:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 10:06:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
Message-ID: <4B4AEA25.7010100@v.loewis.de>

> "Python 2.7 is scheduled to be the last major version in the 2.x
> series released by the Python Software Foundation before it moves into
> 5 years of bugfix-only mode. "
> 
> That doesn't exclude *others* making a Python 2.8 that could include
> all sorts of crazy features. :)
> 
> This is mainly just to get the pointless discussion over with.

I know you are just looking for a compromise, but this shouldn't be it:
the PSF has deliberately stayed out of the actual Python engineering,
so the release that Benjamin makes is not done by the PSF (but by
Benjamin and his contributors :-).

I like the wording as it is (although I find the promise of five years
of bug fix releases a bit too strong).

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 10:19:24 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 10:19:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AEA25.7010100@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
Message-ID: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>

On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I know you are just looking for a compromise, but this shouldn't be it:
> the PSF has deliberately stayed out of the actual Python engineering,
> so the release that Benjamin makes is not done by the PSF (but by
> Benjamin and his contributors :-).

Hm. Yeah. That's right of course. I started with saying "official",
but then I thought "official according to who?". :) So I changed it to
mentioning PSF, but that doesn't work either. I guess the current
writing as as good as it gets, unless we change "scheduled" to
"expected" or something.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Mon Jan 11 10:42:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 Jan 2010 10:42:05 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A5C69.7090007@v.loewis.de> <20100111041754.GB7200@arctrix.com>
Message-ID: <hierpd$dm1$1@ger.gmane.org>

Neil Schemenauer, 11.01.2010 05:17:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time).

Note that there's also the time it takes to make a release (usually 
involving at least one beta release), plus the possibility of introducing 
bugs while adding new features or even while fixing agreed bugs. All of 
this takes time in addition to the time people might want to invest in 
'real' development on the 2.x trunk.

Stefan



From stephen at xemacs.org  Mon Jan 11 10:59:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 11 Jan 2010 18:59:57 +0900
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <8763793qsi.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Schemenauer writes:
 > On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:

 > > If people start taking the carrots we have added to 3.x and
 > > backporting them to keep the 2.x series alive you are essentially
 > > making the 3.x DOA by negating its benefits which I personally
 > > don't agree with.

Well, I think it's *worse* than that, and I don't think you really
mean "DOA", anyway.  (Feel free to correct me, of course.)

The problem I see with backporting lots of stuff, and/or adding new
features that aren't in 3.0, to 2.x is that it will make 2.x even
cruftier, when it was already crufty enough that Guido (and almost all
of python-dev) bit the bullet and said "backward compatibility is no
excuse for keeping something in 3.0".

That surely means that a lot of python-dev denizens will declare
non-support 2.x for x > 7.  It's not going to be the gradual migration
we've seen over the past few months as active people start to spend
more and more time on 3 vs. 2; it will be a watershed.  Especially if
these are new features merged from outside that the "small active
segment" doesn't know anything about.  From the users' point of view,
that amounts to a *fork*, even if it's internal and "friendly".

 > I think we have got to the heart of our disagreement. Assume that
 > some superhuman takes all the backwards compatible goodies from 3.x
 > and merges them into 2.x.

Isn't that a bit ridiculous?  I just don't see any evidence that
anything like that is going to happen.  Worse, if we *assume* it will
happen, I don't see any way to assess whether (1) Python 3 goes belly
up, (2) there's an effective fork confusing the users and draining the
energy of python-dev, or (3) everybody goes "wow!" because it's so
cool that everybody wants to keep maintaining an extra 3 branches
indefinitely.

My opinion is that given the clear direction the "small active
segment" is going, telling the users anything but what Brett proposed
is disinformation.

 > I guess I have more confidence in Python 3 than you do. I don't see
 > why Python 2.x needs to be artificially limited so that Python 3 can
 > benefit.

It's not for Python 3, which you, I, and I'm pretty sure Brett-in-his-
heart-of-hearts agree can take care of itself because it is *better*
than Python 2.  It's for Python, and for the Python community.


From walter at livinglogic.de  Mon Jan 11 11:37:56 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 11:37:56 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001091438.43576.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
Message-ID: <4B4AFF84.7070206@livinglogic.de>

On 09.01.10 14:38, Victor Stinner wrote:

> Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
>>> Good idea, I choosed open(filename, encoding="BOM").
>>
>> On the surface this looks like there's an encoding named "BOM", but
>> looking at your patch I found that the check is still done in
>> TextIOWrapper. IMHO the best approach would to the implement a *real*
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
>> the IO library. It could even be developed as a standalone project and
>> published in the Cheeseshop.
> 
> Why not, this is another solution to the point (2) (Check for a BOM while 
> reading or detect it before?). Which encoding would be used if there is not 
> BOM? UTF-8 sounds like a good choice.

UTF-8 might be a good choice, are the failback could be specified in the
encoding name, i.e.

   open("file.txt", encoding="BOM-UTF-8")

falls back to UTF-8, if there's no BOM at the start.

This could be implemented via a custom codec search function (see
http://docs.python.org/library/codecs.html#codecs.register for more info).

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 12:12:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:12:20 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4AFF84.7070206@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
Message-ID: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>

On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
> UTF-8 might be a good choice

No, fallback if there is no BOM should be the local settings, just as
fallback is today if you don't specify a codec.
I mean, what if you want to look for a BOM but fall back to something
else? How far will we go with encoding special information in the
codecs names? codec='BOM else UTF-16 else locale'? :-)

BOM is not a locale, and should not be a locale. Having a locale
called BOM is wrong per se. It should either be default to look for a
BOM when codec=None, or a special parameter. If none of these are
desired, then we need a special function that takes a filename or file
handle, and looks for a BOM and returns the codec found or None. But
I find that much less natural and obvious than checking the BOM when codec=None.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From regebro at gmail.com  Mon Jan 11 12:13:00 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:13:00 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <319e029f1001110313u1d839deodfe06a6c7ec423cf@mail.gmail.com>

On Mon, Jan 11, 2010 at 12:12, Lennart Regebro <regebro at gmail.com> wrote:
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se.

D'oh! I mean codec here obviously.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 13:29:04 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 13:29:04 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4913DF.7050801@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>
Message-ID: <4B4B1990.90705@livinglogic.de>

On 10.01.10 00:40, "Martin v. L?wis" wrote:
>>> How does the requirement that it be implemented as a codec miss the
>>> point?
>>
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
> 
> Yes - however, Victor currently apparently *doesn't* want it to be the
> default, but wants the user to specify encoding="BOM". If so, it isn't
> the default, and it is easy to implement as a codec.
> 
>>> FWIW, I agree with Walter that if it is provided through the encoding=
>>> argument, it should be a codec. If it is built into the open function
>>> (for whatever reason), it must be provided by some other parameter.
>>
>> Why not simply encoding=None?
> 
> I don't mind. Please re-read Walter's message - it only said that
> *if* this is activated through encoding="BOM", *then* it must be
> a codec, and could be on PyPI. I don't think Walter was talking about
> the case "it is not activated through encoding='BOM'" *at all*.

However if this autodetection feature is useful in other cases (no
matter how it's activated), it should be a codec, because as part of the
open() function it isn't reusable.

>> The default value should provide the most useful
>> behaviour possible. Forcing users to choose between two different autodetection
>> strategies (encoding=None and another one) is a little insane IMO.

And encoding="mbcs" is a third option on Windows.

> That wouldn't disturb me much. There are a lot of things in that area
> that are a little insane, starting with Microsoft Windows :-)

;)

Servus,
   Walter


From barry at python.org  Mon Jan 11 13:37:46 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 07:37:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5DCE.3070909@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com> <4B4A5DCE.3070909@v.loewis.de>
Message-ID: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>

On Jan 10, 2010, at 6:07 PM, Martin v. L?wis wrote:

> As for decisions: I don't think there was an official BDFL pronouncent,
> but I recall Guido posting a message close to that, proposing that 2.7
> will be a release that will see bug fix releases for an indefinite
> period of time (where indefinite != infinite). This was shortly after
> him proposing that perhaps we shouldn't make a 2.7 release at all, and
> stop at 2.6.

IMO, the focus for Python 2.7 (and beyond) must be on helping people transition to Python 3.  That should be the reason why a 2.7 is released and, what should dictate whether a 2.8 is necessary.

Maybe everything people need (except manpower and round tuits) is already there.  I was pleasantly surprised to find that Distribute supports automatically running 2to3 at install time for Python packages.  This allowed me to "port" a recent (admittedly small) package to Python 3 by making it "python2.6 -3" clean and adding a couple of attributes to my setup.py[1].  I don't know how widely that feature's known, but I think it's *huge*, and exactly what we need to be doing.  The more we can make it easy for people to port their Python 2 code to Python 3, and to support that with one code base, the quicker we'll get to a predominantly Python 3 world.

So the question we should be asking is: what's missing from Python 2.7 to help with this transition?  If we can't get it into 2.7, then why, and would pushing it back to 2.8 help at all?

-Barry

[1] modulo a bug in Distribute that caused doctest in separate files to not be used when running  under Python 3.



From solipsis at pitrou.net  Mon Jan 11 13:39:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 13:39:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>  <4B4B1990.90705@livinglogic.de>
Message-ID: <1263213574.3327.0.camel@localhost>


> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

It is reusable as part of io.TextIOWrapper, though.

Regards

Antoine.




From regebro at gmail.com  Mon Jan 11 13:45:30 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 13:45:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
Message-ID: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>

On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

But an autodetect feature is not a codec. Sure it should be reusable,
but making it a codec seems to be  a weird hack to me. And how would
you reuse it if it was a codec? A reusable autodetect feature would be
useable to detect what codec it is. A autodetect codec would not be
useful for that, as it would simply just decode.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 14:21:15 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 14:21:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>	<4B4913DF.7050801@v.loewis.de>
	<4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B25CB.5030003@livinglogic.de>

On 11.01.10 13:45, Lennart Regebro wrote:

> On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
>> However if this autodetection feature is useful in other cases (no
>> matter how it's activated), it should be a codec, because as part of the
>> open() function it isn't reusable.
> 
> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

I think we already had this discussion two years ago in the context of
XML decoding ;):

http://mail.python.org/pipermail/python-dev/2007-November/075138.html

> And how would
> you reuse it if it was a codec? A reusable autodetect feature would be
> useable to detect what codec it is. A autodetect codec would not be
> useful for that, as it would simply just decode.

I have implemented an XML codec (as part of XIST:
http://pypi.python.org/pypi/ll-xist), that can do that:

>>> from ll import xml_codec
>>> import codecs
>>> c = codecs.getincrementaldecoder("xml")()
>>> c.encoding
>>> c.decode("<?xml")
u''
>>> c.encoding
>>> c.decode(" version='1.0'")
u''
>>> c.encoding
>>> c.decode(" encoding='iso-8859-1'?>")
u"<?xml version='1.0' encoding='iso-8859-1'?>"
>>> c.encoding
'iso-8859-1'

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 14:47:12 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 14:47:12 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B25CB.5030003@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B25CB.5030003@livinglogic.de>
Message-ID: <319e029f1001110547n1d1c9831p34b0eac519d13f9d@mail.gmail.com>

On Mon, Jan 11, 2010 at 14:21, Walter D?rwald <walter at livinglogic.de> wrote:
> I think we already had this discussion two years ago in the context of
> XML decoding ;):

Yup. Ans Martins answer then is my answer now:

"> So the code is good, if it is inside an XML parser, and it's bad if it
> is inside a codec?

Exactly so. This functionality just *isn't* a codec - there is no
encoding. Instead, it is an algorithm for *detecting* an encoding."

The conclusion was that a method do autodetect encodings would be
good. I think the same conclusion applies here.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 18:16:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:16:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<4B46F67F.60604@v.loewis.de>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B5CF5.50806@v.loewis.de>

> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

Well, the existing UTF-16 codec also is an autodetect feature (to
detect the endianness), and I don't consider it a weird hack.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 18:27:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:27:01 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B5CF5.50806@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B5CF5.50806@v.loewis.de>
Message-ID: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:16, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> But an autodetect feature is not a codec. Sure it should be reusable,
>> but making it a codec seems to be ?a weird hack to me.
>
> Well, the existing UTF-16 codec also is an autodetect feature (to
> detect the endianness), and I don't consider it a weird hack.

So the BOM codec should raise a UnicodeDecodeError if there is no BOM?
Because that's what it would have to do, in that case, because it
can't fall back on anything, it has to handle and implement all
encodings that have a BOM. And is it then actually very useful? You
would have to do a try/except first with encoding='BOM' and then
encoding=None to get the fallback to the standard.


I must say that I find this whole thing pretty obvious. 'BOM' is not
an encoding. Either there should be a method to get the encoding from
the BOM, returning None of there isn't one, or open() should look at
the BOM when you pass in encoding=None. Or both.

That covers all usecases, is easy and obvious. Either open(file=foo,
encoding=None) or open(file, encoding=encoding_from_bom(file))

I can't see that open(file, encoding='BOM') has any benefit over this,
covers any extra usecase and is clearer in any way. Instead it adds
something confusing: An encoding that isn't an encoding.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From python at mrabarnett.plus.com  Mon Jan 11 18:35:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:35:35 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<201001091438.43576.victor.stinner@haypocalc.com>	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <4B4B6167.40606@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
>> UTF-8 might be a good choice
> 
> No, fallback if there is no BOM should be the local settings, just as
> fallback is today if you don't specify a codec.
> I mean, what if you want to look for a BOM but fall back to something
> else? How far will we go with encoding special information in the
> codecs names? codec='BOM else UTF-16 else locale'? :-)
> 
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se. It should either be default to look for a
> BOM when codec=None, or a special parameter. If none of these are
> desired, then we need a special function that takes a filename or file
> handle, and looks for a BOM and returns the codec found or None. But
> I find that much less natural and obvious than checking the BOM when codec=None.
> 
Or pass a function that accepts a byte stream or the first few bytes and
returns the encoding and any unused bytes (because the byte stream might
not be seekable)?

def guess_encoding(byte_stream):
     data = byte_stream.read(2)
     if data == b"\xFE\xFF":
         return "UTF-16BE", b""
     return "UTF-8", data

text_file = open(filename, encoding=guess_encoding)
...


From python at mrabarnett.plus.com  Mon Jan 11 18:46:30 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:46:30 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
Message-ID: <4B4B63F6.1020309@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>> I know you are just looking for a compromise, but this shouldn't be
>> it: the PSF has deliberately stayed out of the actual Python
>> engineering, so the release that Benjamin makes is not done by the
>> PSF (but by Benjamin and his contributors :-).
> 
> Hm. Yeah. That's right of course. I started with saying "official", 
> but then I thought "official according to who?". :)

"Official" is whatever the BDFL says it is! :-)

> So I changed it to mentioning PSF, but that doesn't work either. I
> guess the current writing as as good as it gets, unless we change
> "scheduled" to "expected" or something.
> 


From martin at v.loewis.de  Mon Jan 11 18:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:59:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>	
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>	
	<4B4B5CF5.50806@v.loewis.de>
	<319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
Message-ID: <4B4B66EC.7000203@v.loewis.de>

> I must say that I find this whole thing pretty obvious. 'BOM' is not
> an encoding.

That I certainly agree with.

> That covers all usecases, is easy and obvious. Either open(file=foo,
> encoding=None) or open(file, encoding=encoding_from_bom(file))
> 
> I can't see that open(file, encoding='BOM') has any benefit over this,

Well, it would have the advantage that Walter pointed out: you can
implement it independent of the open() implementation, and even provide
it in older versions of Python.

Regards,
Martin



From regebro at gmail.com  Mon Jan 11 18:59:52 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:59:52 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B63F6.1020309@mrabarnett.plus.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
Message-ID: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:46, MRAB <python at mrabarnett.plus.com> wrote:
> "Official" is whatever the BDFL says it is! :-)

Heh, right.

So, it should say "Guido wants 2.7 to be the last main version of
Python 2, so it probably will be. We promise to release bugfixes it
for, like, ages".

No need to be needlessly formal. :-D
-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From olemis at gmail.com  Mon Jan 11 19:58:01 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 13:58:01 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
[...]
>

I had similar issues too (please read below ;o) ...

On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>

About guessing the encoding, I experienced this issue while I was
developing a Trac plugin. What I was doing is as follows :

- I guessed the MIME type + charset encoding using Trac MIME API (it
was a CSV file encoded using UTF-16)
- I read the file using `open`
- Then wrapped the file using `codecs.EncodedFile`
- Then used `csv.reader`

... and still get the BOM in the first value of the first row in the CSV file.

{{{
#!python

>>> mimetype
'utf-16-le'
>>> ef = EncodedFile(f, 'utf-8', mimetype)
}}}

IMO I think I am +1 for leaving `open` just like it is, and use module
`codecs` to deal with encodings, but I am strongly -1 for returning
the BOM while using `EncodedFile` (mainly because encoding is
explicitly supplied in ;o)

> --Guido
>

CMIIW anyway ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:


From mal at egenix.com  Mon Jan 11 21:44:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 11 Jan 2010 21:44:34 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
Message-ID: <4B4B8DB2.1060102@egenix.com>

Olemis Lang wrote:
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>> Hi,
>>>
>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>> BOM should be "ignored".
>>>
> [...]
>>
> 
> I had similar issues too (please read below ;o) ...
> 
> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
> 
> About guessing the encoding, I experienced this issue while I was
> developing a Trac plugin. What I was doing is as follows :
> 
> - I guessed the MIME type + charset encoding using Trac MIME API (it
> was a CSV file encoded using UTF-16)
> - I read the file using `open`
> - Then wrapped the file using `codecs.EncodedFile`
> - Then used `csv.reader`
> 
> ... and still get the BOM in the first value of the first row in the CSV file.

You didn't say, but I presume that the charset guessing logic
returned either 'utf-16-le' or 'utf-16-be' - those encodings don't
remove the leading BOM. The 'utf-16' codec will remove the BOM.

> {{{
> #!python
> 
>>>> mimetype
> 'utf-16-le'
>>>> ef = EncodedFile(f, 'utf-8', mimetype)
> }}}

Same here: the UTF-8 codec will not remove the BOM, you have
to use the 'utf-8-sig' codec for that.

> IMO I think I am +1 for leaving `open` just like it is, and use module
> `codecs` to deal with encodings, but I am strongly -1 for returning
> the BOM while using `EncodedFile` (mainly because encoding is
> explicitly supplied in ;o)

Note that EncodedFile() doesn't do any fancy BOM detection or
filtering. This is the job of the codecs.

Also note that BOM removal is only valid at the beginning of
a file. All subsequent BOM-bytes have to be read as-is (they
map to a zero-width non-breaking space) - without removing them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Mon Jan 11 22:12:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 07:12:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <4B4B9447.4060101@gmail.com>

Benjamin Peterson wrote:
 > My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
> 
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

I would call it a documentation bug: by leaving out the __get__ you're
telling Python to override *just* the setting of the attribute, while
still using the normal lookup process for retrieving the attribute (i.e.
allowing the attribute to be shadowed in the instance dictionary).

Adding a "print('Descriptor Invoked')" to the __set__ method in your
example:

>>> x = X()
>>> print(x.attr)
<__main__.Descr object at 0x7f283b51ac50>
>>> x.attr = 42
Descriptor invoked
>>> print(x.attr)
42
>>> x.attr = 6*9
Descriptor invoked
>>> print(x.attr)
54

Note that the behaviour here is still different from that of a data
descriptor: with a data descriptor, once it gets shadowed in the
instance dictionary, the descriptor is ignored *completely*. The only
way to get the descriptor involved again is to eliminate the shadowing.
The non-data descriptor with only __set__ is just choosing not to
override the attribute lookup process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From olemis at gmail.com  Mon Jan 11 22:29:38 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 16:29:38 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B8DB2.1060102@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
	<4B4B8DB2.1060102@egenix.com>
Message-ID: <24ea26601001111329i7f609aa1i9f5db7eb61f1fae7@mail.gmail.com>

Probably one part of this is OT , but I think it could complement the
discussion ;o)

On Mon, Jan 11, 2010 at 3:44 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Olemis Lang wrote:
>>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>>> <victor.stinner at haypocalc.com> wrote:
>>>> Hi,
>>>>
>>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>>> BOM should be "ignored".
>>>>
>> [...]
>>>
>>
>> I had similar issues too (please read below ;o) ...
>>
>> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>>>
>>
>> About guessing the encoding, I experienced this issue while I was
>> developing a Trac plugin. What I was doing is as follows :
>>
>> - I guessed the MIME type + charset encoding using Trac MIME API (it
>> was a CSV file encoded using UTF-16)
>> - I read the file using `open`
>> - Then wrapped the file using `codecs.EncodedFile`
>> - Then used `csv.reader`
>>
>> ... and still get the BOM in the first value of the first row in the CSV file.
>
> You didn't say, but I presume that the charset guessing logic
> returned either 'utf-16-le' or 'utf-16-be'

Yes. In fact they return the full mimetype 'text/csv; charset=utf-16-le' ... ;o)

> - those encodings don't
> remove the leading BOM.

... and they should ?

> The 'utf-16' codec will remove the BOM.
>

In this particular case there's nothing I can do, I have to process
whatever charset is detected in the input ;o)

>> {{{
>> #!python
>>
>>>>> mimetype
>> 'utf-16-le'
>>>>> ef = EncodedFile(f, 'utf-8', mimetype)
>> }}}
>
> Same here: the UTF-8 codec will not remove the BOM, you have
> to use the 'utf-8-sig' codec for that.
>
>> IMO I think I am +1 for leaving `open` just like it is, and use module
>> `codecs` to deal with encodings, but I am strongly -1 for returning
>> the BOM while using `EncodedFile` (mainly because encoding is
>> explicitly supplied in ;o)
>
> Note that EncodedFile() doesn't do any fancy BOM detection or
> filtering.

... directly.

> This is the job of the codecs.
>

OK ... to come back to the scope of the subject, in the general case,
I think that BOM (if any) should be handled by codecs, and therefore
indirectly by EncodedFile . If that's a explicit way of working with
encodings I'd prefer to use that wrapper explicitly in order to
(encode | decode) the file and let the codec detect whether there's a
BOM or not and ?adjust? `tell`, `read` and everything else in that
wrapper (instead of `open`).

> Also note that BOM removal is only valid at the beginning of
> a file. All subsequent BOM-bytes have to be read as-is (they
> map to a zero-width non-breaking space) - without removing them.
>

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Test cases for custom query (i.e report 9) ... PASS (1.0.0)  -
http://simelo.hg.sourceforge.net/hgweb/simelo/trac-gviz/rev/d276011e7297


From martin at v.loewis.de  Mon Jan 11 22:42:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:42:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
Message-ID: <4B4B9B55.1010709@v.loewis.de>

> So the question we should be asking is: what's missing from Python
> 2.7 to help with this transition?

Wrt. the distribute feature you've noticed: Python 3.0 already supports
that in distutils. So from day one of Python 3.x, you could have used
that approach for porting Python code. I had been promoting it ever
since, but it's taking up only slowly, partially because it has
competition in the approaches "burn the bridges" (i.e. convert to 3.x
once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
(i.e. try to write code that works in both 2.x and 3.x unmodified).

> If we can't get it into 2.7, then
> why, and would pushing it back to 2.8 help at all?

I've done a fair bit of 3.x porting, and I'm firmly convinced that
2.x can do nothing:

a) telling people that they have to move to 2.6 first actually
   hurts migration, instead of helping, because it implies to them
   that they have to drop old versions (e.g. 2.3.) - just because
   they had *always* dropped old versions before supporting new ones.
b) IMO, people also don't gain much by first migrating to 2.6.
   In principle, it gives them the opportunity to get py3k warnings.
   However, I haven't heard a single positive report where these
   warnings have actually helped people in porting. Yours is the
   first report saying that you followed the official guideline,
   but you didn't state whether doing so actually helped (or whether
   you just ported to 2.6 because the guideline told you to).
c) whatever 2.7 does (except perhaps for the warnings), it won't
   be useful to applications, because they couldn't use it, anyway:
   they need to support 2.4 and 2.5, and it won't have any of the
   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
   might actually hurt porting: people may have to special-case
   2.7 because their work-arounds for older versions may break in 2.7
   (e.g. testing whether a name is *not* defined, when it becomes
   defined in 2.7), plus it gives them an incentive to not port
   yet until they can drop support for anything before 2.7.

Inherently, 2.8 can't improve on that.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 22:44:24 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:44:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
Message-ID: <4B4B9BB8.3070505@v.loewis.de>

> So, it should say "Guido wants 2.7 to be the last main version of
> Python 2, so it probably will be. We promise to release bugfixes it
> for, like, ages".
> 
> No need to be needlessly formal. :-D

That summarizes my understanding of what Guido said fairly well :-)

+1 for putting it into the announcements.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Mon Jan 11 23:11:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 11 Jan 2010 22:11:44 +0000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4B9447.4060101@gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com>
Message-ID: <4B4BA220.20701@voidspace.org.uk>

On 11/01/2010 21:12, Nick Coghlan wrote:
> Benjamin Peterson wrote:
>   >  My question is: Is this a doc bug or a implementation bug? If the
>    
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>>      
> [snip...]
>
> Note that the behaviour here is still different from that of a data
> descriptor: with a data descriptor, once it gets shadowed in the
> instance dictionary, the descriptor is ignored *completely*. The only
> way to get the descriptor involved again is to eliminate the shadowing.
> The non-data descriptor with only __set__ is just choosing not to
> override the attribute lookup process.
>
>    

Does that mean we need a third class of descriptors that are neither 
data descriptors nor non-data descriptors?

What should we call them: really-not-data-descriptors?

All the best,

Michael

> Cheers,
> Nick.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From guido at python.org  Mon Jan 11 23:55:01 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Jan 2010 14:55:01 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9BB8.3070505@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de> 
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com> 
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com> 
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com> 
	<4B4B9BB8.3070505@v.loewis.de>
Message-ID: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>

+1 from me. (With the caveat that "time will tell" is definitely the
ultimate answer. Plans may change unexpectedly, even though we don't
expect them to.)

PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
helpful. But I trust he has ported a lot more code to 3.x than I have
personally. Are there other experiences?

--Guido

On Mon, Jan 11, 2010 at 1:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So, it should say "Guido wants 2.7 to be the last main version of
>> Python 2, so it probably will be. We promise to release bugfixes it
>> for, like, ages".
>>
>> No need to be needlessly formal. :-D
>
> That summarizes my understanding of what Guido said fairly well :-)
>
> +1 for putting it into the announcements.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Tue Jan 12 00:16:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 00:16:47 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <4B4BB15F.5020807@v.loewis.de>

Guido van Rossum wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
> 
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. 

That's really because I haven't even attempted to use it. Instead, the
software would fall flat on its own when running the test suite in 3.x.
IOW, I don't need 2.6 to tell me what might break when I can see 3.x
actually breaking.

Regards,
Martin


From david.lyon at preisshare.net  Tue Jan 12 00:22:42 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:22:42 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>


Hi Martin,

> Of course, the less active fraction of Python contributors may not
> notice, since they just chose to not contribute (which, of course,
> is fine). However, asking me to work twice as much as I want to
> on the project to keep two branches alive is just unfair.

Totally true. Actually as an end-developer I'd say that python
2.x series from a programming perspective is quite good. It
doesn't need the addition of a lot of new features imho.

So for me, I think too much time spent there would be not
yield great benefits.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Well, we all know your work is super quality. :-) That's not
being contested.

However, Quality can be measured different ways and it can
be assessed in different ways. Quality itself is a subjective
thing.

The point I'm only making is that if a piece of software
doesn't have "new" things added over time, then users
can get a reverse impression of a lack of quality.

We've all seen where 'internal' quality can increase
and user perceptions can decrease.

It could be things like improved graphics and things readily
apparent to the user.

At the moment, I would say that the "internal" quality
of the python 2.x series is super high. "external" quality
issues such as the packaging dilemma give the user the
opposite quality experience. But people are working on
that as best they can elsewhere. I'll leave it at that.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Python 3.x needs more carrots.

>From an ordinary (perphaps ignorant) user perspective there
is nothing. Yes, we know if we actually will start
programming then we will like it more.

But my wishes to Santa Claus would be allow the free
flow of PEPs for Python 3 packaging. Even encourage
it.

As an end developer, here's what I'd like from Santa
in 2010 to get me to swap to python 3:

 * get all the packages on pypi tested for python 3

 * put a web based package manager in python 3. This
   would perhaps be based around PIP (keep many
   people happy) and would look much like the built
   in web-console that you get with a $200 router.

 * Incorporate SCM based end-user software installs
   by adding to python3-stdlib the SCM support
   packages (cvs,bzr,svn,hg). This would *really*
   help in the scientific community.

 * put a web interface on distutils also so that
   we don't have to use a command line interface.
   (I want a pic of a smiley girl to great me
    when I build something - "Are you ready
    to build now Honey?"). ok - I joke. But the
    point is made.

So, ok, maybe these things aren't about 'code'
quality. But rather user experience.

Things like these do count also as "quality"
via the technical term "perception of quality".

If the PEP process is as unblocked as the
documentation implies, implying that anybody
can contribute to Python 3. Then there shouldn't
be any issue.

David









From solipsis at pitrou.net  Tue Jan 12 00:37:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 23:37:47 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100112T003506-611@post.gmane.org>

David Lyon <david.lyon <at> preisshare.net> writes:
> 
> > This has nothing to do with pushing 3.x, but all with managing
> > available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As someone who experiences the difference almost every day, I can say 3.x
definitely has enough carrots for me. The simple fact that I will be able to
raise exceptions with non-ASCII error messages without having to workaround a
hopelessly broken conversion/reporting logic is a huge improvement. And that's
only a small part of the picture.

Regards

Antoine.




From janssen at parc.com  Tue Jan 12 00:47:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 11 Jan 2010 15:47:55 PST
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <17215.1263253675@parc.com>

> David Lyon <david.lyon <at> preisshare.net> writes:
> > 
> > > This has nothing to do with pushing 3.x, but all with managing
> > > available manpower and still providing quality software.
> > 
> > Python 3.x needs more carrots.

I'd be happy to move UpLib to Python 3, when the various packages that I
need are also on Python 3.  And that depresses me to think about.  I
need

   Medusa
   ReportLab
   PyLucene
   Email
   Vobject
   Mutagen
   Pyglet
   Hachoir
   Win32

I'm on the mailing lists for a lot of these, and the only one that I
know is thinking about Python 3 is Email.  I'd expect Win32 and PIL to
also be thinking about it, but I haven't heard anything.

Bill


From david.lyon at preisshare.net  Tue Jan 12 00:47:51 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:47:51 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <1220.218.214.45.58.1263253671.squirrel@syd-srv02.ezyreg.com>

Antoine writes:

> As someone who experiences the difference almost every day, I can say 3.x
> definitely has enough carrots for me. The simple fact that I will be able
> to
> raise exceptions with non-ASCII error messages without having to
> workaround a
> hopelessly broken conversion/reporting logic is a huge improvement. And
> that's
> only a small part of the picture.

In no way could I disagree with you.

Ascii works fine for us here - but I guess we're lucky.

Just keep pushing python 3 and have a nice day..

David





From barry at python.org  Tue Jan 12 01:11:28 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:11:28 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100111191128.39020d89@freewill>

On Jan 11, 2010, at 10:42 PM, Martin v. L?wis wrote:

>Wrt. the distribute feature you've noticed: Python 3.0 already supports
>that in distutils. So from day one of Python 3.x, you could have used
>that approach for porting Python code. I had been promoting it ever
>since, but it's taking up only slowly, partially because it has
>competition in the approaches "burn the bridges" (i.e. convert to 3.x
>once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
>(i.e. try to write code that works in both 2.x and 3.x unmodified).

Interesting.  The only reason I never used it before is because I didn't know
about it.  Am I alone?

Maybe I'm projecting my own preferences, but it seems to me that supporting
both Python 2 and 3 from the same code base would be a very powerful way to
enable a large amount of existing code to support Python 3.  I'm certainly
going to do this from now on with all the libraries I maintain.  I don't have
any cycles to write code twice and I can't abandon Python 2 yet.  I'm
skeptical that code can work unmodified in both 2 and 3 without 2to3.

As an example, the one library I've already ported used a metaclass.  I don't
see any way to specify that the metaclass should be used in a portable way.
In Python 2.6 it's:

class Foo:
    __metaclass__ = Meta

and in Python 3 it's:

class Foo(metaclass=Meta):

2to3 made that pain go away.

>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>2.x can do nothing:
>
>a) telling people that they have to move to 2.6 first actually
>   hurts migration, instead of helping, because it implies to them
>   that they have to drop old versions (e.g. 2.3.) - just because
>   they had *always* dropped old versions before supporting new ones.

Is it just an implication, or is it reality?  I have yet to try to support
older versions of Python and also support Python 3.  (The one package I've
done this with so far only supports Python 2.6.)

>b) IMO, people also don't gain much by first migrating to 2.6.
>   In principle, it gives them the opportunity to get py3k warnings.
>   However, I haven't heard a single positive report where these
>   warnings have actually helped people in porting. Yours is the
>   first report saying that you followed the official guideline,
>   but you didn't state whether doing so actually helped (or whether
>   you just ported to 2.6 because the guideline told you to).

Python 2.6 has other useful features, which I want to take advantage of, so
getting py3k warnings is a bonus.  Running 'python2.6 -3 setup.py test' over
my code did in fact help clean up a couple of problems.  Seems like a good
first step when considering Python 3 support to me.

>c) whatever 2.7 does (except perhaps for the warnings), it won't
>   be useful to applications, because they couldn't use it, anyway:
>   they need to support 2.4 and 2.5, and it won't have any of the
>   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
>   might actually hurt porting: people may have to special-case
>   2.7 because their work-arounds for older versions may break in 2.7
>   (e.g. testing whether a name is *not* defined, when it becomes
>   defined in 2.7), plus it gives them an incentive to not port
>   yet until they can drop support for anything before 2.7.
>
>Inherently, 2.8 can't improve on that.

I'm not so sure.  Yes, as a package maintainer there are older versions to
think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
released, what will be the minimum version of Python provided by most OS
vendors (where the majority of Python users probably get their 'python')?  I
guess some people will have to support everything from Python 2.3 to 2.8 but
you're talking supporting something like a spread of 7 years of Python
versions.  What other platform do you support for 7 years?  Even Ubuntu long
term support (LTS) releases only live for 5 years and even then, newer Pythons
are often back ported.  Or if not, then who cares?  You're not going to use a
newer version of a library on an LTS anyway.

I'm not necessarily saying that justifies a 2.8 release.  I'm just asking how
we can make it easier for people to port to Python 3.  The automatic 2to3 has
already made it easier for me to port one library to Python 3 because I barely
had to even think about it.  Maybe that's enough.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/840169eb/attachment-0003.pgp>

From barry at python.org  Tue Jan 12 01:12:19 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:12:19 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <20100111191219.5fdd2542@freewill>

On Jan 11, 2010, at 02:55 PM, Guido van Rossum wrote:

>PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
>helpful. But I trust he has ported a lot more code to 3.x than I have
>personally. Are there other experiences?

I've only done one small library so far, but it was helpful to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/ba78a57a/attachment-0003.pgp>

From andrew at bemusement.org  Tue Jan 12 01:07:26 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 12 Jan 2010 11:07:26 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100112000726.GO32351@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> I've done a fair bit of 3.x porting, and I'm firmly convinced that
> 2.x can do nothing:
[...]
> Inherently, 2.8 can't improve on that.

I agree that there are limitations like the ones you've listed, but I
disagree with your conclusion.  Maybe you assume that it's just as hard
to move to 2.8 (using the py3k backports not available in say 2.5) as it
is to 3.x?

But a hypothetical 2.8 would also give people a way to move closer to
py3k without giving up on using all their 2.x-only dependencies.  I
think it's much more likely that libraries like Twisted can support 2.8
in the near future than 3.x.

Then, when all of your dependencies (or viable alternatives to those
dependencies) are available for 3.x, you'll have an easier transition if
you can start from a 2.x with fewer differences in features.

Fundamentally the more 2.x can converge on 3.x, the easier it will be
for users to make the leap, because it will be a smaller leap.  The
longer the 2.x series lives, the more these newer 2.x versions like 2.7
and maybe even 2.8 will be available on common platforms for people to
depend upon as minimum versions, which means that as time goes by they
can depend on a version that's closer to 3.x.  And so again, the leap
becomes easier to make.  So to me it's pretty clear that 2.8 *can*
improve the transition path to 3.x.  It may or may not be a worthwhile
use of effort for python-dev to make 2.8, but that's different to saying
it's inherently pointless.

-Andrew.


From solipsis at pitrou.net  Tue Jan 12 01:28:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 00:28:26 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <loom.20100112T012550-387@post.gmane.org>

Andrew Bennetts <andrew <at> bemusement.org> writes:
> 
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies.  I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

I don't think a 2.8 could make you much closer to 3.x. AFAICT, every possible
way to ease transition to 3.x will already be in 2.7, or almost. But perhaps I'm
lacking imagination.

Regards

Antoine.




From brian.curtin at gmail.com  Tue Jan 12 02:57:46 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 11 Jan 2010 19:57:46 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>

On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:

> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
>
> -Brett
>
I think it would be interesting to see how people are using the tracker, or
how they want to be using it. For example, there are currently over 1500
open issues with no stage set, some of which seemingly haven't been read by
anyone at all. Would a properly set stage field save issues from falling
into a black hole?

Food for thought: according to the last tracker summary, there are over 1000
open issues with a patch, and issues stay open an average of 700 days
(median 450).

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/a3439436/attachment-0003.htm>

From jackdied at gmail.com  Tue Jan 12 04:53:24 2010
From: jackdied at gmail.com (Jack Diederich)
Date: Mon, 11 Jan 2010 22:53:24 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>

On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

[sidebar]
1) the metaclass fixer was a PITA to implement.
2) 95% of __metaclass__ definitions searchable via google code were of
the "__metaclass__ = type" variety.  The 2to3 patch exists only
because of the few other uses.
3) 100% of the module level assignments in public projects were the
"__metaclass__ = type" variety which is why there isn't a fixer for
that.  Also, a fixer would have been really, really ugly (munge every
class definition in this module because there is a top level
assignment).

-Jack


From benjamin at python.org  Tue Jan 12 05:01:40 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 Jan 2010 22:01:40 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <1afaf6161001112001t5e7bab83t7d93f33fa11ce849@mail.gmail.com>

2010/1/11 Jack Diederich <jackdied at gmail.com>:
> On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
>> As an example, the one library I've already ported used a metaclass. ?I don't
>> see any way to specify that the metaclass should be used in a portable way.
>> In Python 2.6 it's:
>>
>> class Foo:
>> ? ?__metaclass__ = Meta
>>
>> and in Python 3 it's:
>>
>> class Foo(metaclass=Meta):
>>
>> 2to3 made that pain go away.
>
> [sidebar]
> 1) the metaclass fixer was a PITA to implement.

Does this make it any less useful, though? :)



-- 
Regards,
Benjamin


From steven.bethard at gmail.com  Tue Jan 12 06:57:18 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 11 Jan 2010 21:57:18 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>

On Mon, Jan 11, 2010 at 4:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

Actually there's a solution to this one too:

    FooBase = Meta('FooBase', (), {})
    class Foo(FooBase):
        ...

That should work in Python 2.X and 3.X.

I've got argparse running on Python 2.3-3.1, and the changes were
pretty easy. You can see them all in the revision here:

    http://code.google.com/p/argparse/source/detail?r=12

I have aspirations of putting all of the tricks I learned up up on the
Wiki somewhere, but I just haven't had the time.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From regebro at gmail.com  Tue Jan 12 07:30:10 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:30:10 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>

On Mon, Jan 11, 2010 at 23:55, Guido van Rossum <guido at python.org> wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
>
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. But I trust he has ported a lot more code to 3.x than I have
> personally. Are there other experiences?

It doesn't warn for that many of the unportable problems, but I'm not
sure it can warn for them either. It's certainly helpful, just not
very much. :) I think the biggest help was added in 2.6.2, and that's
warning for old integer division. It will also warn for modules that
aren't supported anymore, if I remember correctly, and that's often
helpful. But it won't warn for real tricky problems, like binary vs
text in strings, and I don't see how it can either.

And, I just realized, it doesn't warn for you using cmp or __cmp__
either, and 2to3 won't fix that, so it should actually warn for it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:49:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:49:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:11, Barry Warsaw <barry at python.org> wrote:
> Interesting. ?The only reason I never used it before is because I didn't know
> about it. ?Am I alone?

Maybe not, but the Distribute feature is there because IMO the
distutils feature by itself isn't particularily useful. You need to
write your own distutils extensions, in practice, and they are not
trivial. Distribute has simply done it for you. :)

> I'm skeptical that code can work unmodified in both 2 and 3 without 2to3.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:53:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:53:20 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <319e029f1001112253x511f8109ja2300a022010ef99@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:07, Andrew Bennetts <andrew at bemusement.org> wrote:
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. ?I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

When 2.7 was discussed several people agreed that 2.7 should include
as much 3.x stuff as possible to ease transition. That turned out to
not be very much, so I'm not sure there is more. :)

Unless of course, 2.8 starts including more of the refactored
libraries, but that's a very minor issue in porting, so it won't help
much.

To really help, it needs to start implement things that break
backwards compatibility. That would be weird. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ncoghlan at gmail.com  Tue Jan 12 10:39:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:39:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4BA220.20701@voidspace.org.uk>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com> <4B4BA220.20701@voidspace.org.uk>
Message-ID: <4B4C435B.7080903@gmail.com>

Michael Foord wrote:
>> Note that the behaviour here is still different from that of a data
>> descriptor: with a data descriptor, once it gets shadowed in the
>> instance dictionary, the descriptor is ignored *completely*. The only
>> way to get the descriptor involved again is to eliminate the shadowing.
>> The non-data descriptor with only __set__ is just choosing not to
>> override the attribute lookup process.
>>
>>    
> 
> Does that mean we need a third class of descriptors that are neither
> data descriptors nor non-data descriptors?

Not really - leaving out __get__ when defining __set__ just creates a
data descriptor that happens to use the default attribute look-up
process rather than defining a different one.

(Note that I had the data/non-data terminology backwards in my previous
message - I tend to think of the two kinds of descriptor as enforced and
non-enforced respectively precisely because I have to think about it to
remember that "functions are non-data descriptors and properties are
data descriptors, hence non-data descriptors only define __get__ while
data descriptors define __set__". I don't find the data/non-data
terminology intuitive at all)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:44:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:44:18 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>	<4B4B9BB8.3070505@v.loewis.de>	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
	<319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
Message-ID: <4B4C4472.10104@gmail.com>

Lennart Regebro wrote:
> And, I just realized, it doesn't warn for you using cmp or __cmp__
> either, and 2to3 won't fix that, so it should actually warn for it.

I have a vague recollection that we tried to warn for that and ended up
nixing the warning due to vast swarms of false alarms (or because you
couldn't write correct 2.x code without triggering it).

I'd be happy for someone to prove my recollection wrong though (i.e. by
writing a patch that implements such warnings).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:48:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:48:57 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4C4589.70906@gmail.com>

David Lyon wrote:
>> This has nothing to do with pushing 3.x, but all with managing
>> available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As Guido has said a few times, the gains are far greater for *new*
Python developers than they are for existing ones.

Existing developers have to unlearn old habits and wait for libraries
they already use to get ported. New developers can just get started with
a much cleaner language. They don't have as rich a 3rd party ecosystem
on Python 3 as they would on Python 2.x at this point in time, but
unlike existing developers they also have the luxury of cherry-picking
just those packages that already have Python 3 support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Jan 12 11:20:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:20:27 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <hihida$unc$1@ger.gmane.org>

Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit?:
> 
> For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all.

I think most issues /have/ been read. It's just that for many of them, 
nobody is interested enough in or feels competent enough for fixing them.

> Would a properly set stage field save issues from
> falling into a black hole?

I don't think this has anything to do with properly setting the stage 
field. We just have limited time and manpower. Perhaps one of our goals 
should be to reach out more to potential contributors.

> Food for thought: according to the last tracker summary, there are over
> 1000 open issues with a patch, and issues stay open an average of 700
> days (median 450).

As for open issues with a patch, a "code review" button sending this 
patch to Rietveld (or any other similar tool) is something many of us 
have been hoping for :-) It would make reviewing easier, faster and more 
thorough (because you visualize things better than by looking at a diff).

Regards

Antoine.



From solipsis at pitrou.net  Tue Jan 12 11:36:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:36:49 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <hihjc1$45m$1@ger.gmane.org>

Le Tue, 12 Jan 2010 10:20:27 +0000, Antoine Pitrou a ?crit?:
> 
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.

Speaking of which, Steve had something to say about it:
http://holdenweb.blogspot.com/2009/11/comments-or-not-public-or-
private.html

cheers

Antoine.



From ncoghlan at gmail.com  Tue Jan 12 13:10:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 22:10:14 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <4B4C66A6.2040601@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
>> For example, there are currently over
>> 1500 open issues with no stage set, some of which seemingly haven't been
>> read by anyone at all.
> 
> I think most issues /have/ been read. It's just that for many of them, 
> nobody is interested enough in or feels competent enough for fixing them.

There are actually a whole host of reasons issues can stagnate:
- a feature request may seem reasonable (hence it doesn't get rejected
outright), but the right API may not be clear (hence it doesn't get
implemented in the near term)
- a patch may be reviewed and found to have significant defects (or
simply be overreaching the stated goal) but the original patch poster
loses interest after meeting resistance in their ambition to fix
something that is "obviously" broken
- a problem may simply be hard to fix in a backwards compatible way (or
even at all!)

Ultimately it boils down to Antoine's two categories (lack of interest
and lack of relevant expertise to make a final decision) but there are a
lot of different ways to get into those two buckets.

Because we aren't ruthless about pruning those kinds of issues out of
the tracker they're the ones that are going to accumulate over time.

I'd actually be interested to know what the issue stats are like when
RFEs are excluded and when the search is limited to the items flagged as
'easy'. If easy bug fix issues are taking a long time to get closed than
that would bother me a lot more than RFEs that sit on the tracker for years.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 12 13:14:47 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:14:47 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <20100112071447.675c8f24@freewill>

On Jan 11, 2010, at 10:53 PM, Jack Diederich wrote:

>3) 100% of the module level assignments in public projects were the
>"__metaclass__ = type" variety which is why there isn't a fixer for
>that.  Also, a fixer would have been really, really ugly (munge every
>class definition in this module because there is a top level
>assignment).

And almost certainly unnecessary.  IME, those are all to easily make bare
class definitions new style in Python 2.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/29b5ff24/attachment-0003.pgp>

From barry at python.org  Tue Jan 12 13:16:09 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:16:09 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
Message-ID: <20100112071609.1dcfffa6@freewill>

On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:

>Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
>That should work in Python 2.X and 3.X.

Ugly, but good call! :)

>I've got argparse running on Python 2.3-3.1, and the changes were
>pretty easy. You can see them all in the revision here:
>
>    http://code.google.com/p/argparse/source/detail?r=12
>
>I have aspirations of putting all of the tricks I learned up up on the
>Wiki somewhere, but I just haven't had the time.

The more resources we can provide people, both in code and in documentation,
the better.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/a2ba5b3e/attachment-0003.pgp>

From fuzzyman at voidspace.org.uk  Tue Jan 12 13:29:12 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 12:29:12 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112071609.1dcfffa6@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
	<20100112071609.1dcfffa6@freewill>
Message-ID: <4B4C6B18.7050008@voidspace.org.uk>

On 12/01/2010 12:16, Barry Warsaw wrote:
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>    
>> Actually there's a solution to this one too:
>>
>>     FooBase = Meta('FooBase', (), {})
>>     class Foo(FooBase):
>>         ...
>>
>> That should work in Python 2.X and 3.X.
>>      
> Ugly, but good call! :)
>
>    

There are all sorts of tricks. For example you can do exception handling 
that works with pre-2.6 syntax and 3.0 with a bare except and using 
sys.exc_info. It is horrible, but acceptable for short pieces of code (I 
have a couple of small modules that do this).

I haven't yet tried converting larger code-bases to Python 3, but I 
think the workflow advocated by Martin is greatly preferable to the 
hacks and tricks needed to make the same codebase run under 2 & 3.

Michael

>> I've got argparse running on Python 2.3-3.1, and the changes were
>> pretty easy. You can see them all in the revision here:
>>
>>     http://code.google.com/p/argparse/source/detail?r=12
>>
>> I have aspirations of putting all of the tricks I learned up up on the
>> Wiki somewhere, but I just haven't had the time.
>>      
> The more resources we can provide people, both in code and in documentation,
> the better.
>
> Thanks!
> -Barry
>    
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/7acd54f3/attachment-0003.htm>

From mal at egenix.com  Tue Jan 12 14:09:50 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 12 Jan 2010 14:09:50 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <4B4C749E.4040009@egenix.com>

Brett Cannon wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> 
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> * argparse (PEP 389)
> * brief mention on still wanting to break out the stdlib from CPython
> * an official policy on extension modules? (i.e. must have a pure Python
> implementation which can mean a ctypes implementation unless given an
> explicit waiver)
> 
> That's everything from a stdlib perspective. I have half-baked ideas, but
> nothing concrete enough to bring up unless people really want to discuss
> stuff like how to potentially standardize module deprecation warnings, etc.
> 
> In terms of me personally, I do plan to bring up at some point during the
> summit these points which don't squarely fit during my time slot:
> 
> * an official unofficial policy on how new proposed features should come to
> us (i.e. working code to python-ideas with a shell of a PEP that includes
> abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
> 
> If there is something missing from the stdlib discussion that you think
> should be brought up at the summit let me know. And if there is something
> here you want to discuss before the summit let me know and I can start a
> separate thread on it.

Could you please put these things and the results up on the Python
wiki ?!

We're going to have a language summit at EuroPython this year
as well and may want to continue/extend the discussion based
on what you're doing at PyCon.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From brian.curtin at gmail.com  Tue Jan 12 15:33:06 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 12 Jan 2010 08:33:06 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <cf9f31f21001120633n6460b55as6064228cce41c949@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:20, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
> >
> > For example, there are currently over
> > 1500 open issues with no stage set, some of which seemingly haven't been
> > read by anyone at all.
>
> I think most issues /have/ been read. It's just that for many of them,
> nobody is interested enough in or feels competent enough for fixing them.
>
> > Would a properly set stage field save issues from
> > falling into a black hole?
>
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.


Agreed, I didn't mean to place blame on the stage field, I just ran with how
I view that field since it was mentioned. When I'm thinking in "code-writing
developer mode" (tm), I'm more likely to go to issues which have a stage so
I know what I'm going into -- what needs to be worked on. When I'm in
project cleanup mode, I go by stageless issues -- what is necessary for this
to begin or end work. Maybe others work differently...software projects take
all kinds :-)

Anyways, sorry for the off-topic. If this is a summit worthy discussion,
cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/cf466340/attachment-0003.htm>

From rdmurray at bitdance.com  Tue Jan 12 17:12:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 11:12:28 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C66A6.2040601@gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org> <4B4C66A6.2040601@gmail.com>
Message-ID: <20100112161228.300EA1D688D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 22:10:14 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There are actually a whole host of reasons issues can stagnate:
> - a feature request may seem reasonable (hence it doesn't get rejected
> outright), but the right API may not be clear (hence it doesn't get
> implemented in the near term)
> - a patch may be reviewed and found to have significant defects (or
> simply be overreaching the stated goal) but the original patch poster
> loses interest after meeting resistance in their ambition to fix
> something that is "obviously" broken
> - a problem may simply be hard to fix in a backwards compatible way (or
> even at all!)

I would add:

- a patch is reviewed but the reviewer requests a second opinion before
  committing, which never arrives, and the original reviewer forgets
  about the issue.

I have occasionally observed apparently moribund issues spring to life
and get resolved when a triage person does something as simple as updating
the releases to which an issue applies.

> Ultimately it boils down to Antoine's two categories (lack of interest
> and lack of relevant expertise to make a final decision) but there are a
> lot of different ways to get into those two buckets.

There's a third bucket here: lack of time.  Sometimes there is interest
and expertise, but no time.  Worse, sometimes we end up waiting on the
person with the expertise and interest but no time, when someone else
with somewhat less expertise but more time should just go ahead and do
the commit.  (*That* one should be fixable.)

> Because we aren't ruthless about pruning those kinds of issues out of
> the tracker they're the ones that are going to accumulate over time.

Another other category that hangs around in the tracker is items for
which there simply isn't a consensus.  I suppose those could be brought
to python-dev for a final decision if someone wanted to tackle that task.

And I don't think it is a lack of ruthlessness.  I think it is the current
community consensus that we leave these issues open in the tracker in
case someone does come along and want to deal with them. (*)

> I'd actually be interested to know what the issue stats are like when
> RFEs are excluded and when the search is limited to the items flagged as
> 'easy'. If easy bug fix issues are taking a long time to get closed than
> that would bother me a lot more than RFEs that sit on the tracker for
> years.

I'd also be interested to know what the average and median lifetime
of closed bug reports is.  Not the metrics for open issues, but the
metrics for closed ones.  My theory is that we close a lot of bugs fairly
promptly, but the ones in the above categories make the average age of
*open* issues high.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls

(*) For example, I'm currently going through all the open issues
relating to the email package, and some of them are pretty darn
old, yet still have useful content.


From brett at python.org  Tue Jan 12 18:40:13 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:40:13 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C749E.4040009@egenix.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<4B4C749E.4040009@egenix.com>
Message-ID: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>

On Tue, Jan 12, 2010 at 05:09, M.-A. Lemburg <mal at egenix.com> wrote:

> Brett Cannon wrote:
> > Michael has given me the hg transition/stdlib time slot at the language
> > summit this year. In regards to that I plan to lead a discussion on:
> >
> > * where we are at w/ the Hg transition (Dirkjan should be there and I did
> a
> > blog post on this topic recently:
> > http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> > * argparse (PEP 389)
> > * brief mention on still wanting to break out the stdlib from CPython
> > * an official policy on extension modules? (i.e. must have a pure Python
> > implementation which can mean a ctypes implementation unless given an
> > explicit waiver)
> >
> > That's everything from a stdlib perspective. I have half-baked ideas, but
> > nothing concrete enough to bring up unless people really want to discuss
> > stuff like how to potentially standardize module deprecation warnings,
> etc.
> >
> > In terms of me personally, I do plan to bring up at some point during the
> > summit these points which don't squarely fit during my time slot:
> >
> > * an official unofficial policy on how new proposed features should come
> to
> > us (i.e. working code to python-ideas with a shell of a PEP that includes
> > abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> > * any changes needed to the issue tracker to help with the workflow?
> (stage
> > field seems like a failed experiment and we now have several effective
> > triage people who can help w/ guiding changes)
> >
> > If there is something missing from the stdlib discussion that you think
> > should be brought up at the summit let me know. And if there is something
> > here you want to discuss before the summit let me know and I can start a
> > separate thread on it.
>
> Could you please put these things and the results up on the Python
> wiki ?!
>
> We're going to have a language summit at EuroPython this year
> as well and may want to continue/extend the discussion based
> on what you're doing at PyCon.
>
>
I expect there will be at least summary emails on what gets discussed. There
is also a chance that it will be videotaped.

-Brett


> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 12 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/04964610/attachment-0003.htm>

From brett at python.org  Tue Jan 12 18:47:50 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:47:50 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>

On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
>
>>  * any changes needed to the issue tracker to help with the workflow?
>> (stage field seems like a failed experiment and we now have several
>> effective triage people who can help w/ guiding changes)
>>
>> -Brett
>>
> I think it would be interesting to see how people are using the tracker, or
> how they want to be using it. For example, there are currently over 1500
> open issues with no stage set, some of which seemingly haven't been read by
> anyone at all. Would a properly set stage field save issues from falling
> into a black hole?
>
>
"Using" is the key word there. I know this thread went on about how bugs
tend to end up being left open, but what I was proposing to talk about was
whether there is any shift desired by the seasoned tracker users to help in
their work. For instance, the Stage field was meant to help, but I don't
think it is really getting used much, which makes it at best just a UI junk
and at worst something to confuse new users. So I just wanted to discuss
things as a group to see if we could streamline some things, add others,
etc.

At worst I will try to grab people like Mark D., R. David, Antoine, Georg,
etc. at the summit (not sure exactly which the heavy bug closers will be
there), take them aside, and flat-out ask what they need to make their jobs
easier.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/3f808530/attachment-0003.htm>

From brett at python.org  Tue Jan 12 19:29:06 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:29:06 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C6B18.7050008@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com> 
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org> 
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill> 
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com> 
	<20100112071609.1dcfffa6@freewill> <4B4C6B18.7050008@voidspace.org.uk>
Message-ID: <bbaeab101001121029v28a863ccg8213ed494f15160c@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 12/01/2010 12:16, Barry Warsaw wrote:
>
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>
>
>  Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
> That should work in Python 2.X and 3.X.
>
>
>  Ugly, but good call! :)
>
>
>
>
> There are all sorts of tricks. For example you can do exception handling
> that works with pre-2.6 syntax and 3.0 with a bare except and using
> sys.exc_info. It is horrible, but acceptable for short pieces of code (I
> have a couple of small modules that do this).
>
> I haven't yet tried converting larger code-bases to Python 3, but I think
> the workflow advocated by Martin is greatly preferable to the hacks and
> tricks needed to make the same codebase run under 2 & 3.
>
>
In other words we need to pull together a HOWTO for Python source like the
one for extension modules that Benjamin wrote and make it rather prominently
linked from the Python 3 documentation index page.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/86cc3a21/attachment-0003.htm>

From rdmurray at bitdance.com  Tue Jan 12 19:31:23 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 13:31:23 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
Message-ID: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> >>  * any changes needed to the issue tracker to help with the workflow?
> >> (stage field seems like a failed experiment and we now have several
> >> effective triage people who can help w/ guiding changes)
> >>
> > I think it would be interesting to see how people are using the tracker, or
> > how they want to be using it. For example, there are currently over 1500
> > open issues with no stage set, some of which seemingly haven't been read by
> > anyone at all. Would a properly set stage field save issues from falling
> > into a black hole?
> >
> "Using" is the key word there. I know this thread went on about how bugs
> tend to end up being left open, but what I was proposing to talk about was
> whether there is any shift desired by the seasoned tracker users to help in
> their work. For instance, the Stage field was meant to help, but I don't
> think it is really getting used much, which makes it at best just a UI junk
> and at worst something to confuse new users. So I just wanted to discuss
> things as a group to see if we could streamline some things, add others,
> etc.

Well, I for one find the stage field useful.  It reminds me at a glance
where the issue is at in the workflow (and 'not set' is a valid place
in the workflow that an issue sometimes stays in for a while for reason
other than not having been triaged yet).  I suspect that if we discussed
it as a group (we who make the most changes on the tracker) we could
improve it, and the interface in general.

By the way, you could talk to people who aren't going to be at the
summit on #python-dev; I think all the currently tracker-active people
hang out there on a regular basis.

I'll have to give some thought to what changes/improvements might be
most useful, now that I've been doing this for a while.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Tue Jan 12 19:34:02 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:34:02 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com> 
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com> 
	<20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
Message-ID: <bbaeab101001121034m6de86385u4e0e72301e404a59@mail.gmail.com>

On Tue, Jan 12, 2010 at 10:31, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> > On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> > >>  * any changes needed to the issue tracker to help with the workflow?
> > >> (stage field seems like a failed experiment and we now have several
> > >> effective triage people who can help w/ guiding changes)
> > >>
> > > I think it would be interesting to see how people are using the
> tracker, or
> > > how they want to be using it. For example, there are currently over
> 1500
> > > open issues with no stage set, some of which seemingly haven't been
> read by
> > > anyone at all. Would a properly set stage field save issues from
> falling
> > > into a black hole?
> > >
> > "Using" is the key word there. I know this thread went on about how bugs
> > tend to end up being left open, but what I was proposing to talk about
> was
> > whether there is any shift desired by the seasoned tracker users to help
> in
> > their work. For instance, the Stage field was meant to help, but I don't
> > think it is really getting used much, which makes it at best just a UI
> junk
> > and at worst something to confuse new users. So I just wanted to discuss
> > things as a group to see if we could streamline some things, add others,
> > etc.
>
> Well, I for one find the stage field useful.  It reminds me at a glance
> where the issue is at in the workflow (and 'not set' is a valid place
> in the workflow that an issue sometimes stays in for a while for reason
> other than not having been triaged yet).  I suspect that if we discussed
> it as a group (we who make the most changes on the tracker) we could
> improve it, and the interface in general.
>
> By the way, you could talk to people who aren't going to be at the
> summit on #python-dev; I think all the currently tracker-active people
> hang out there on a regular basis.
>
>
Sounds reasonable.


> I'll have to give some thought to what changes/improvements might be
> most useful, now that I've been doing this for a while.


How about everyone who is active on bugs.python.org give it some thought and
we can try to have a discussion at some point in the relatively near future.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/26a09128/attachment-0003.htm>

From ncoghlan at gmail.com  Tue Jan 12 21:58:49 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 06:58:49 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<4B4C749E.4040009@egenix.com>
	<bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
Message-ID: <4B4CE289.6040709@gmail.com>

Brett Cannon wrote:
> I expect there will be at least summary emails on what gets discussed.
> There is also a chance that it will be videotaped.

The Wiki makes for a better summary archive though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Tue Jan 12 22:53:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:53:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
Message-ID: <4B4CEF3D.8000107@v.loewis.de>

>> a) telling people that they have to move to 2.6 first actually
>>   hurts migration, instead of helping, because it implies to them
>>   that they have to drop old versions (e.g. 2.3.) - just because
>>   they had *always* dropped old versions before supporting new ones.
> 
> Is it just an implication, or is it reality?

That's only the implication. However, this was precisely the dialogue
when talking to Django. If you start with "start supporting 2.6", the
immediate response, without listening further, was, "ok, wait until we
drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).

Then explain it to the individual you are talking to, wait for the next
developer of the project step along, and see how he brings up the
very same line of thinking (supporting new versions == dropping support
for old versions).

I think only part of that comes from the maintenance burden. The other
part is that they *want* to drop support for old versions, so that they
can eventually start using new features (e.g. generator expressions).
So they welcome the requirement to support new versions as an excuse
to drop old ones ("it is obvious that you have to drop 2.3 to support
3.2"). However, their users then won't let them drop old versions.


> 
>> b) IMO, people also don't gain much by first migrating to 2.6.
>>   In principle, it gives them the opportunity to get py3k warnings.
>>   However, I haven't heard a single positive report where these
>>   warnings have actually helped people in porting. Yours is the
>>   first report saying that you followed the official guideline,
>>   but you didn't state whether doing so actually helped (or whether
>>   you just ported to 2.6 because the guideline told you to).
> 
> Python 2.6 has other useful features, which I want to take advantage of

I think you are a minority with that, being able to actually use the 2.6
features already. Many projects can't, as they have to support at least
2.4 still (so the with statement is right out).

>> Inherently, 2.8 can't improve on that.
> 
> I'm not so sure.  Yes, as a package maintainer there are older versions to
> think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
> released, what will be the minimum version of Python provided by most OS
> vendors (where the majority of Python users probably get their 'python')?

"Current" Linux distributions will have 2.6 then. "Old" installations
will have 2.4.

> I
> guess some people will have to support everything from Python 2.3 to 2.8 but
> you're talking supporting something like a spread of 7 years of Python
> versions.  What other platform do you support for 7 years?

I think 2.3 will really be gone by the time 2.8 might get released. Even
with 2.7, you'd end up with a span of seven years, though.

Python had been supporting Windows 95 for more than 7 years (I think
rather 9 or 10), likewise Windows 3.1 before that. Python 2.7 will
likely still support Windows 2000, which then will be ten years old.

Solaris support will probably go back to Solaris 2.6, which will be
13 years old when Python 2.7 gets released.

It's only the Linux (and OS X) releases that move so quickly.

Regards,
Martin


From martin at v.loewis.de  Tue Jan 12 22:56:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:56:55 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
Message-ID: <4B4CF027.4080007@v.loewis.de>

> Maybe not, but the Distribute feature is there because IMO the
> distutils feature by itself isn't particularily useful. You need to
> write your own distutils extensions, in practice, and they are not
> trivial.

I wouldn't say that. My Django port works with bare distutils (as
does Django itself), and it works fine.

That distribute had to redo it is only because setuptools *replaces*
the build_py command, as does the 2to3 support in distutils. So only
if you have a different build_py already, you can't use what is in
distutils.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Tue Jan 12 23:02:34 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:02:34 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CEF3D.8000107@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>
	<4B4CEF3D.8000107@v.loewis.de>
Message-ID: <4B4CF17A.1000503@voidspace.org.uk>

On 12/01/2010 21:53, "Martin v. L?wis" wrote:
>>> a) telling people that they have to move to 2.6 first actually
>>>    hurts migration, instead of helping, because it implies to them
>>>    that they have to drop old versions (e.g. 2.3.) - just because
>>>    they had *always* dropped old versions before supporting new ones.
>>>        
>> Is it just an implication, or is it reality?
>>      
> That's only the implication. However, this was precisely the dialogue
> when talking to Django. If you start with "start supporting 2.6", the
> immediate response, without listening further, was, "ok, wait until we
> drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).
>
> Then explain it to the individual you are talking to, wait for the next
> developer of the project step along, and see how he brings up the
> very same line of thinking (supporting new versions == dropping support
> for old versions).
>
> I think only part of that comes from the maintenance burden. The other
> part is that they *want* to drop support for old versions, so that they
> can eventually start using new features (e.g. generator expressions).
> So they welcome the requirement to support new versions as an excuse
> to drop old ones ("it is obvious that you have to drop 2.3 to support
> 3.2"). However, their users then won't let them drop old versions.
>
>
>    

I agree with Martin that the *perception* is that to use Python 2.6 to 
help you port to Python 3 you have to be willing to drop support for 
earlier versions of Python.

>>      
>>> b) IMO, people also don't gain much by first migrating to 2.6.
>>>    In principle, it gives them the opportunity to get py3k warnings.
>>>    However, I haven't heard a single positive report where these
>>>    warnings have actually helped people in porting. Yours is the
>>>    first report saying that you followed the official guideline,
>>>    but you didn't state whether doing so actually helped (or whether
>>>    you just ported to 2.6 because the guideline told you to).
>>>        
>> Python 2.6 has other useful features, which I want to take advantage of
>>      
> I think you are a minority with that, being able to actually use the 2.6
> features already. Many projects can't, as they have to support at least
> 2.4 still (so the with statement is right out).
>
>    
Well, the IronPython community has almost completely moved over to 
IronPython 2.6. :-)

We tend to ship our Python runtime with our applications though. As it 
happens I'm now working with CPython on the server (Python 2.5) and 
IronPython in the browser where we are using 2.6. The new property 
feature is nice, as is having with without a __future__ import.

All the best,


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From regebro at gmail.com  Tue Jan 12 23:03:41 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 23:03:41 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF027.4080007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
	<4B4CF027.4080007@v.loewis.de>
Message-ID: <319e029f1001121403x14ef7ec8x729fb80fa67feaef@mail.gmail.com>

On Tue, Jan 12, 2010 at 22:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Maybe not, but the Distribute feature is there because IMO the
>> distutils feature by itself isn't particularily useful. You need to
>> write your own distutils extensions, in practice, and they are not
>> trivial.
>
> I wouldn't say that. My Django port works with bare distutils (as
> does Django itself), and it works fine.
>
> That distribute had to redo it is only because setuptools *replaces*
> the build_py command, as does the 2to3 support in distutils. So only
> if you have a different build_py already, you can't use what is in
> distutils.

Yeah, you are right, I misremembered. The actual additional feature is
the support for the test command. Testing under Python 2 and 3 without
it is annoying.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan 12 23:04:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:04:37 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <4B4CF1F5.4050403@v.loewis.de>

> [...]
>> I've done a fair bit of 3.x porting, and I'm firmly convinced that
>> 2.x can do nothing:
> [...]
>> Inherently, 2.8 can't improve on that.
> 
> I agree that there are limitations like the ones you've listed, but I
> disagree with your conclusion.  Maybe you assume that it's just as hard
> to move to 2.8 (using the py3k backports not available in say 2.5) as it
> is to 3.x?

Not at all, no. I'd rather expect that code that runs on 2.7 will run
on 2.8 unmodified.

> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. 

How so? If they use anything that is new in 2.8, they *will* need to
drop support for anything before it, no???

> I think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
that help Twisted in moving to 3.2?

> Then, when all of your dependencies (or viable alternatives to those
> dependencies) are available for 3.x, you'll have an easier transition if
> you can start from a 2.x with fewer differences in features.

But you won't *have* fewer differences. Just because your code runs
on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
for that). This doesn't get you any closer - you can't use any of
the 2.8 features as long as you have to support older versions of
Python.

> Fundamentally the more 2.x can converge on 3.x, the easier it will be
> for users to make the leap, because it will be a smaller leap.

No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
likely won't.

> The
> longer the 2.x series lives, the more these newer 2.x versions like 2.7
> and maybe even 2.8 will be available on common platforms for people to
> depend upon as minimum versions, which means that as time goes by they
> can depend on a version that's closer to 3.x.

No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
released in 2010. Then, in 2015, it may be that everybody has migrated
to 2.7, which then is a common platform.

If you release 2.8 in 2012, then, in 2015, people will be split between
2.7 and 2.8, and so there won't be a common platform before 2017.

So stopping 2.x development *earlier* will also give us a common
platform earlier.

Regareds,
Martin


From martin at v.loewis.de  Tue Jan 12 23:07:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:07:02 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <4B4CF286.5040002@v.loewis.de>

> I think it would be interesting to see how people are using the tracker,
> or how they want to be using it. For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all. Would a properly set stage field save issues from
> falling into a black hole?

I personally don't think this would be the case.

What would help is people who actually *work* on the issues, rather than
merely reading them. However, this being open source, there is no way to
force it (unless you create an incentive, such as the five-for-one
deal).

Regards,
Martin


From python at mrabarnett.plus.com  Tue Jan 12 23:10:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 12 Jan 2010 22:10:28 +0000
Subject: [Python-Dev] regex module
Message-ID: <4B4CF354.2050603@mrabarnett.plus.com>

Hi all,

I'm back on the regex module after doing other things and I'd like your
opinion on a number of matters:

Firstly, the current re module has a bug whereby it doesn't split on
zero-width matches. The BDFL has said that this behaviour should be
retained by default in case any existing software depends on it. My
question is: should my regex module still do this for Python 3?
Speaking personally, I'd like it to behave correctly, and Python 3 is
the version where backwards-compatibility is allowed to be broken.

Secondly, Python 2 is reaching the end of the line and Python 3 is the
future. Should I still release a version that works with Python 2? I'm
thinking that it could be confusing if new regex module did zero-width
splits correctly in Python 3 but not in Python 2. And also, should I
release it only for Python 3 as a 'carrot'?

Finally, the module allows some extra backslash escapes, eg \g<name>, in
the pattern. Should it treat ill-formed escapes, eg \g, as it would have
treated them in the re module?

Thanks


From michael at voidspace.org.uk  Tue Jan 12 23:46:49 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:46:49 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
Message-ID: <4B4CFBD9.1090009@voidspace.org.uk>

I presume the email below is about the Windows binary. Does the AMD64 
release work on intel 64bit and can we make the wording clearer on the 
download page?

The current description is " Windows AMD64 binary".

All the best,

Michael

-------- Original Message --------
Subject: 	Download Page - AMD64
Date: 	Fri, 11 Dec 2009 04:03:16 -0600
From: 	Thomas Brownback <thomas.brownback at gmail.com>
To: 	webmaster at python.org



Should I use the AMD64 version of Python on an Intel 64 chip? I know 
those 64-bit implementations are very similar, but are they similar 
enough that your AMD64 will work on Intel?

If so, perhaps a note should be placed on that page to help avoid 
confusion. Explicit being better than implicit and all.

Thanks for your time,
Thomas

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/80d6b539/attachment-0003.htm>

From andrew at bemusement.org  Tue Jan 12 23:49:56 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Wed, 13 Jan 2010 09:49:56 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100112224956.GC28576@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > But a hypothetical 2.8 would also give people a way to move closer to
> > py3k without giving up on using all their 2.x-only dependencies. 
> 
> How so? If they use anything that is new in 2.8, they *will* need to
> drop support for anything before it, no???
> 
> > I think it's much more likely that libraries like Twisted can support 2.8
> > in the near future than 3.x.
> 
> Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
> that help Twisted in moving to 3.2?

I'm not talking about Twisted moving to 3.x (FWIW, I think the only
movement there so far is some patches for some -3 warnings).  The
situation I'm describing is a project X that:

  (a) has 2.x-only dependencies, and
  (b) would like to be as close as possible to 3.x (because they like
      the new features and/or want to be as ready as possible to jump
      when (a) is fixed).

So just because project X depends on e.g. Twisted, and that Twisted in
turn still supports 2.4, doesn't mean that X cannot move to 2.8, and
doesn't mean it would get no benefit from doing so.

[...]
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.

But this is my point.  I think they would as an intermediate step to
jumping to 3.x (which also requires dropping 2.5, after all!), if for
some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

-Andrew.



From tjreedy at udel.edu  Tue Jan 12 23:51:31 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 17:51:31 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <hiiudf$2uv$1@ger.gmane.org>

On 1/12/2010 5:04 PM, "Martin v. L?wis" wrote:

> But you won't *have* fewer differences. Just because your code runs
> on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
> for that). This doesn't get you any closer - you can't use any of
> the 2.8 features as long as you have to support older versions of
> Python.
>
>> Fundamentally the more 2.x can converge on 3.x, the easier it will be
>> for users to make the leap, because it will be a smaller leap.
>
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.
>
>> The
>> longer the 2.x series lives, the more these newer 2.x versions like 2.7
>> and maybe even 2.8 will be available on common platforms for people to
>> depend upon as minimum versions, which means that as time goes by they
>> can depend on a version that's closer to 3.x.
>
> No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
> released in 2010. Then, in 2015, it may be that everybody has migrated
> to 2.7, which then is a common platform.
>
> If you release 2.8 in 2012, then, in 2015, people will be split between
> 2.7 and 2.8, and so there won't be a common platform before 2017.

Just like people today may need to work with both 2.5 and 2.6, or 
privately backport 2.6 bugfixes to 2.5.

> So stopping 2.x development *earlier* will also give us a common
> platform earlier.

With years of bug fixes and hence high quality.




From tjreedy at udel.edu  Wed Jan 13 00:05:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 18:05:12 -0500
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <hiiv75$5md$1@ger.gmane.org>

On 1/12/2010 5:10 PM, MRAB wrote:
> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.

Are you writing a new module with a new name? If so, do you expect it to 
replace or augment re? (This is the same question as for optparse vs. 
argparse, which I understand to not yet be decided.)
>
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?

2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
2.7 stdlib is already out. A new engine should get some community 
testing before going in the stdlib. Even 3.2 beta is not that far off 
(8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?

> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?

What does re do with analogous cases?

Terry Jan Reedy



From david.lyon at preisshare.net  Wed Jan 13 00:20:54 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 13 Jan 2010 10:20:54 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C4589.70906@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<4B4C4589.70906@gmail.com>
Message-ID: <1333.218.214.45.58.1263338454.squirrel@syd-srv02.ezyreg.com>

> Nick wrote:
>>> This has nothing to do with pushing 3.x, but all with managing
>>> available manpower and still providing quality software.
>>
>> Python 3.x needs more carrots.
>
> As Guido has said a few times, the gains are far greater for *new*
> Python developers than they are for existing ones.

Well both you and Guido are most likely 100% correct. :-)

> They don't have as rich a 3rd party ecosystem
> on Python 3 as they would on Python 2.x at this point in time, but
> unlike existing developers they also have the luxury of cherry-picking
> just those packages that already have Python 3 support.

Most likely. I wouldn't want to say anything to discourage
people from going to python 3. In other languages, I have
much experience of making the jump to 'a whole new world'.

It's unsettling at first, but after that, you suddenly
find the new model has a better turbo than the last and
you're at the next corner faster than you can think. So
it's all good.

But I still maintain Python 3.0 needs more carrots. For example,
if mercurial or any other cool lib gets added to python 3 (and
I can name a few that should go in python 3) then they should
be added to python 3 and not to python 2.x

They would serve as good carrots.

Make fresh the python 3 stdlib and preserve the python 2.x stdlib.

I really think we are somewhat short on resources to do
what Guido has asked about bringing python up to CPAN level
with respect to packages.

We're starting a long way back with horses and swords and
trying to catch a well fed and greased modern machine..

I'm sure we can get a modern package testbot operational
for python.

But I wish those who were complaining about the packaging
problem so much could throw some money at the problem
instead of moaning about it. As is done on other open-source
projects. Some organisation would be beneficial.

Finding funds so that a small group of people could
work on the problem would be a great boost to forward
progress. I just think python packaging needs six
months of that to repair the years and years of neglect
and stagnation. Even if it is only beer and bus fare
money. It just needs a temporary shot of adrenalin in
the arm.. so to speak..

Regards

David











From martin at v.loewis.de  Wed Jan 13 00:28:14 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 00:28:14 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D058E.4030404@v.loewis.de>

> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?

"intel 64bit" is as clear as mud. It could mean the "Intel 64"
architecture, or it could mean the "IA-64" architecture, both
are 64-bit architectures with processors manufactured by Intel.
The former is officially called AMD64 (not by Intel, but
officially), and our AMD64 binaries run on such processors.
The latter is also called "Itanium", and we stopped producing
binaries for that architecture a while ago; the AMD64 binaries
will *not* run on it.

The wording could probably be changed to match the reader's
expectation more, most likely, it would also get more incorrect
in the process. To make it correct and explicit, an entire
paragraph educating users about 64-bit architectures and that
there are many of them and that they are mutually incompatible
might be required.

Most likely, Thomas' processor implements the AMD64 architecture,
even though it is manufactured by Intel. A short sentence that
he would probably understand (given that he used "Intel 64", not
"64bit") is:

"""The binaries for AMD64 will also work on processors that implement
the Intel 64 architecture (formerly EM64T), i.e. the architecture that
Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
They will not work on Intel Itanium Processors (formerly IA-64)."""

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 00:30:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 00:30:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112224956.GC28576@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>	<4B4CF1F5.4050403@v.loewis.de>
	<20100112224956.GC28576@steerpike.home.puzzling.org>
Message-ID: <4B4D0613.1010503@v.loewis.de>

> I'm not talking about Twisted moving to 3.x (FWIW, I think the only
> movement there so far is some patches for some -3 warnings).  The
> situation I'm describing is a project X that:
> 
>   (a) has 2.x-only dependencies, and
>   (b) would like to be as close as possible to 3.x (because they like
>       the new features and/or want to be as ready as possible to jump
>       when (a) is fixed).

This assumes that jumping to 3.x is easier if you are closer to it.
Please trust me that this is not the case.

>> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
>> likely won't.
> 
> But this is my point.  I think they would as an intermediate step to
> jumping to 3.x (which also requires dropping 2.5, after all!), if for
> some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

No, and no. No: it's not an intermediate step, and no, supporting 3.x
does *not* require dropping 2.5.

Regards,
Martin



From fuzzyman at voidspace.org.uk  Wed Jan 13 00:40:35 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:40:35 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D058E.4030404@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
Message-ID: <4B4D0873.5070006@voidspace.org.uk>

On 12/01/2010 23:28, "Martin v. L?wis" wrote:
> [snip...]
> """The binaries for AMD64 will also work on processors that implement
> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
> They will not work on Intel Itanium Processors (formerly IA-64)."""
>
>    

To those not intimately aware of the history and present of 64 bit 
architecture this sentence would be a great addition - thanks. I'm 
adding it now as a footnote.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From lists at cheimes.de  Wed Jan 13 00:41:04 2010
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 13 Jan 2010 00:41:04 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D0890.2030505@cheimes.de>

Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64 
> release work on intel 64bit and can we make the wording clearer on the 
> download page?
> 
> The current description is " Windows AMD64 binary".

The installer works on all AMD64 compatible Intel CPUs. *scnr*

As you most likely know all modern Intel 64bit CPUs are based on AMD's
x86-64 design. Only the Itanium family is based on the other Intel 64bit
design: IA-64. The name AMD64 was chosen because most Linux and BSD
distributions call it so. The name 'AMD64' has caused confusion in the
past because users thought that the installer works only on AMD CPUs.

How about:

 * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
X86-64 binary -- does not include source)

instead of:

 * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
not include source)

?

Christia


From fuzzyman at voidspace.org.uk  Wed Jan 13 00:55:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:55:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0873.5070006@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
	<4B4D0873.5070006@voidspace.org.uk>
Message-ID: <4B4D0BD4.5050109@voidspace.org.uk>

On 12/01/2010 23:40, Michael Foord wrote:
> On 12/01/2010 23:28, "Martin v. L?wis" wrote:
>> [snip...]
>> """The binaries for AMD64 will also work on processors that implement
>> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
>> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
>> They will not work on Intel Itanium Processors (formerly IA-64)."""
>>
>
> To those not intimately aware of the history and present of 64 bit 
> architecture this sentence would be a great addition - thanks. I'm 
> adding it now as a footnote.
>

I can add it now to the main download page and existing release pages - 
but are there changes needed to any scripts to change pages created for 
*new* releases?

All the best,

Michael Foord

> All the best,
>
> Michael Foord
>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From python at mrabarnett.plus.com  Wed Jan 13 01:22:01 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 00:22:01 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <hiiv75$5md$1@ger.gmane.org>
References: <4B4CF354.2050603@mrabarnett.plus.com> <hiiv75$5md$1@ger.gmane.org>
Message-ID: <4B4D1229.70708@mrabarnett.plus.com>

Terry Reedy wrote:
> On 1/12/2010 5:10 PM, MRAB wrote:
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
> 
> Are you writing a new module with a new name? If so, do you expect it to 
> replace or augment re? (This is the same question as for optparse vs. 
> argparse, which I understand to not yet be decided.)
 >
It's a module called 'regex'. It can be used in place of 're' by using
"import regex as re", except for differences such as "\g<name>" being a
legal group reference in pattern strings.
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
> 
> 2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
> 2.7 stdlib is already out. A new engine should get some community 
> testing before going in the stdlib. Even 3.2 beta is not that far off 
> (8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?
> 
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
> 
> What does re do with analogous cases?
> 
The 're' module treats r"\g" as "g"; both 're' and 'regex' treat, say, 
r"\q" as "q". The closest analogue to what I'm asking about is that re
treats the ill-formed repeat r"x{1," as a literal, which sort of
suggests that r"\g" should be treated as "g", but r"\g<name>" is now a
group reference (re would treat that as "g<name>". Does that sound
reasonable?


From fuzzyman at voidspace.org.uk  Wed Jan 13 01:50:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 00:50:39 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D18DF.6070502@voidspace.org.uk>

On 12/01/2010 23:41, Christian Heimes wrote:
> Michael Foord wrote:
>    
>> I presume the email below is about the Windows binary. Does the AMD64
>> release work on intel 64bit and can we make the wording clearer on the
>> download page?
>>
>> The current description is " Windows AMD64 binary".
>>      
> The installer works on all AMD64 compatible Intel CPUs. *scnr*
>
> As you most likely know all modern Intel 64bit CPUs are based on AMD's
> x86-64 design. Only the Itanium family is based on the other Intel 64bit
> design: IA-64. The name AMD64 was chosen because most Linux and BSD
> distributions call it so. The name 'AMD64' has caused confusion in the
> past because users thought that the installer works only on AMD CPUs.
>
> How about:
>
>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
>
> instead of:
>
>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)
>    
Right - I've made that change for the Python 2.6, 2.7, 3.0 and 3.1 
download pages with a footnote. Prior to that we were offering ia64 
*and* amd64 releases so I've left those unchanged.

Thanks,

Michael

> ?
>
> Christia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From exarkun at twistedmatrix.com  Wed Jan 13 02:40:03 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Jan 2010 01:40:03 -0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100113014003.1898.1305834328.divmod.xquotient.219@localhost.localdomain>

On 12 Jan, 10:04 pm, martin at v.loewis.de wrote:
>>[...]
>>>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>>>2.x can do nothing:
>>[...]
>>>Inherently, 2.8 can't improve on that.
>>
>>I agree that there are limitations like the ones you've listed, but I
>>disagree with your conclusion.  Maybe you assume that it's just as 
>>hard
>>to move to 2.8 (using the py3k backports not available in say 2.5) as 
>>it
>>is to 3.x?
>
>Not at all, no. I'd rather expect that code that runs on 2.7 will run
>on 2.8 unmodified.
>>But a hypothetical 2.8 would also give people a way to move closer to
>>py3k without giving up on using all their 2.x-only dependencies.
>
>How so? If they use anything that is new in 2.8, they *will* need to
>drop support for anything before it, no???
>>I think it's much more likely that libraries like Twisted can support 
>>2.8
>>in the near future than 3.x.
>
>Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
>that help Twisted in moving to 3.2?

I'm not reading this thread carefully enough to make any arguments on 
either side, but I can contribute a fact.

Twisted very likely does not support 2.8 today.  I base this on the fact 
that Twisted does not support 2.7 today, and I expect 2.8 will be more 
like 2.7 than it will be like 2.3 - 2.6 (which Twisted does support).

When I say "support" here, I mean "all of the Twisted unit tests pass on 
it".

Jean-Paul


From tseaver at palladion.com  Wed Jan 13 03:57:46 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 12 Jan 2010 21:57:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <4B4D36AA.7020607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Karen Tracey wrote:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>>
> Well yay.  Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.  That's
> down from 104 failures and 40 errors with 2.7 alpha 1.
> 
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Just to add another "success" data point:  Zope2's trunk, as well as the
2.12 release, passes all tests (2535 on the trunk) and brings up the
appserver just fine under 2.7a2.

There is an obnoxious deprecation warning out of the distutils:

  DeprecationWarning: 'compiler' specifies the compiler type in
  build_ext. If you want to get the compiler object itself, use
  'compiler_obj'

which is likely a simple one-line fix, if I only knew what the hell it
was whining about. ;)  The warning is extra obnoxious because it doesn't
tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
argument).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktNNqQACgkQ+gerLs4ltQ7d5gCcCGKVjyOlxnrAln0UnRibS7kd
lNIAoIs1RlSGMtJWaY11BqptfDmQvR87
=mIOO
-----END PGP SIGNATURE-----



From python at mrabarnett.plus.com  Wed Jan 13 04:09:34 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 03:09:34 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <4B4D396E.4050500@mrabarnett.plus.com>

MRAB wrote:
> Hi all,
> 
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
> 
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
> 
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
> 
> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
> 
I've just noticed something odd about the re module: the sub() method
doesn't take 'pos' or 'endpos' arguments. search() does; match() does;
findall() does(); finditer() does; but sub() doesn't. Maybe there has
never been a demand for it. (Nor split(), for that matter.)


From brett at python.org  Wed Jan 13 04:58:11 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 19:58:11 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>

On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:

> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
>
>
If it is a separate module under a different name it can do the proper
thing. People will just need to be aware of the difference when they import
the module.


> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
>
>
That's totally up to you. There is practically no chance of it getting into
the 2.x under the stdlib at this point since 2.7b1 is coming up and this
module has not been out in the wild for a year (to my knowledge).  If you
want to support 2.x that's fine and I am sure users would appreciate it, but
it isn't necessary to get into the Python 3 stdlib.


> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
>
>
If you want to minimize the differences then it should probably match. As I
said, since it is a different name to import under it can deviate where
reasonable, just make sure to clearly document the deviations.

-Brett



> Thanks
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/f1f73d56/attachment-0003.htm>

From martin at v.loewis.de  Wed Jan 13 07:11:49 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 07:11:49 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D6425.3060307@v.loewis.de>

> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
> 
> instead of:
> 
>  * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)

-1. AMD doesn't want us to use the term x86-64 anymore, but wants us
to use AMD64 instead. I think we should comply - they invented the
architecture, so they have the right to give it a name. Neither
Microsoft nor Intel have such a right.

Regards,
Martin


From sridharr at activestate.com  Wed Jan 13 07:47:38 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 12 Jan 2010 22:47:38 -0800
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D6C8A.7070307@activestate.com>

On 1/12/2010 2:46 PM, Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?
>
> The current description is " Windows AMD64 binary".

FWIW, we simply use (64-bit, x64).

   Platform	          Download
   ==============================================
   Windows (x86)	          Windows Installer (MSI)
   Windows (64-bit, x64)	  Windows Installer (MSI)

https://www.activestate.com/activepython/downloads/

-srid


From solipsis at pitrou.net  Wed Jan 13 08:08:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 13 Jan 2010 07:08:55 +0000 (UTC)
Subject: [Python-Dev] Fwd: Download Page - AMD64
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <loom.20100113T080717-468@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)

+1. I don't care about trademarks or official names, we should call it whatever
is obvious for our users.
As for Itanium, it is practically dead, I think there is little chance of people
mixing it up with x86-64.

cheers

Antoine.




From michael at voidspace.org.uk  Wed Jan 13 10:32:00 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 09:32:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <4B4D9310.6060907@voidspace.org.uk>

On 13/01/2010 06:11, "Martin v. L?wis" wrote:
>> How about:
>>
>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
>>
>> instead of:
>>
>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>> not include source)
>>      
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.
>    

I think we should use whatever is most informative and least confusing 
to our users - we owe our allegiance to them and not to a processor vendor.

Michael


> Regards,
> Martin
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ncoghlan at gmail.com  Wed Jan 13 11:30:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:30:50 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF17A.1000503@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk>
Message-ID: <4B4DA0DA.7070007@gmail.com>

Michael Foord wrote:
> I agree with Martin that the *perception* is that to use Python 2.6 to
> help you port to Python 3 you have to be willing to drop support for
> earlier versions of Python.

Note that increased 3.x compatibility in the most recent 2.x release
will always help in two scenarios:

1. New projects that want to use 2.x only libraries but want to be ready
for the Py3k transition in their own code (e.g. new 2.7 features like
set literals, dict and set comprehensions and the view* dict methods can
be translated far more cleanly by 2to3 than the closest comparable 2.6
code).

2. Similarly new projects that use a 3.x trunk can be more readily
translated to a 2.7 version with 3to2, whereas some constructs may be
difficult to recreate in earlier Python versions. I would expect this to
be significantly less common then the first scenario though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:38:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:38:31 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <loom.20100113T080717-468@post.gmane.org>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<loom.20100113T080717-468@post.gmane.org>
Message-ID: <4B4DA2A7.2080408@gmail.com>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> How about:
>>
>>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
> 
> +1. I don't care about trademarks or official names, we should call it whatever
> is obvious for our users.

Until this discussion, I didn't even know the architecture had any name
other than x86-64.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:39:40 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:39:40 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4D36AA.7020607@palladion.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com>
Message-ID: <4B4DA2EC.3050908@gmail.com>

Tres Seaver wrote:
> There is an obnoxious deprecation warning out of the distutils:
> 
>   DeprecationWarning: 'compiler' specifies the compiler type in
>   build_ext. If you want to get the compiler object itself, use
>   'compiler_obj'
> 
> which is likely a simple one-line fix, if I only knew what the hell it
> was whining about. ;)  The warning is extra obnoxious because it doesn't
> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
> argument).

Could you kick a tracker issues in Tarek's direction for that one?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From vinay_sajip at yahoo.co.uk  Wed Jan 13 13:24:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 13 Jan 2010 12:24:09 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <loom.20100113T131816-515@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> If there is something missing from the stdlib discussion that you think should
be brought up at the summit let me know. And if there is something here you want
to discuss before the summit let me know and I can start a separate thread on it.

I'm sorry I won't be able to come to the language summit, but I would like if
possible to expedite a pronouncement on PEP 391 (configuring logging using
dictionaries). I believe I addressed all the comments made on the discussion
threads mentioned in the PEP and so I'm not sure what more I need to do to get a
pronouncement. I guess the stdlib slot gives an opportunity for people to air
their views and so I'd be grateful if you added it to the agenda.

Thanks & regards,

Vinay Sajip



From murman at gmail.com  Wed Jan 13 14:35:51 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 07:35:51 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>

On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.

I see and agree with the motivation behind your point, but it's just
as reasonable to mimic the name Microsoft uses: the release is for
Windows rather than the processor. On MSDN Microsoft uses
parentheticals x86, ia64 and x64; this would suggest a name like
Python 2.6.4 installer for Windows (x64). Unfortunately this usage
doesn't seem to be reflected in consumer-oriented product pages, so
I'm uncertain how clear it is for those downloading Python.

-- 
Michael Urman


From ncoghlan at gmail.com  Wed Jan 13 15:04:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 00:04:28 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
Message-ID: <4B4DD2EC.3030709@gmail.com>

Michael Urman wrote:
> On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
> 
> I see and agree with the motivation behind your point, but it's just
> as reasonable to mimic the name Microsoft uses: the release is for
> Windows rather than the processor. On MSDN Microsoft uses
> parentheticals x86, ia64 and x64; this would suggest a name like
> Python 2.6.4 installer for Windows (x64). Unfortunately this usage
> doesn't seem to be reflected in consumer-oriented product pages, so
> I'm uncertain how clear it is for those downloading Python.

As Windows doesn't run on non-x86 architectures, their naming is
generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Linux, on the other hand, can run on multiple 64 bit architectures, so
being more specific and using the official AMD64 nomenclature makes sense.

In this case, making it clear that the 64-bit Windows binaries work for
both Intel and AMD chips is more important than using only the official
architecture name.

Cheers,
Nick.

P.S. This discussion made me realise that out of habit I had dropped the
32-bit version of Python on my new 64-bit Windows box...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tseaver at palladion.com  Wed Jan 13 17:49:55 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 13 Jan 2010 11:49:55 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA2EC.3050908@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com> <4B4DA2EC.3050908@gmail.com>
Message-ID: <4B4DF9B3.4030403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> There is an obnoxious deprecation warning out of the distutils:
>>
>>   DeprecationWarning: 'compiler' specifies the compiler type in
>>   build_ext. If you want to get the compiler object itself, use
>>   'compiler_obj'
>>
>> which is likely a simple one-line fix, if I only knew what the hell it
>> was whining about. ;)  The warning is extra obnoxious because it doesn't
>> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
>> argument).
>
> Could you kick a tracker issues in Tarek's direction for that one?

Done:

  http://bugs.python.org/issue7694


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktN+bMACgkQ+gerLs4ltQ6s4gCdENhtVQWzoH449BCDz8SNx/4s
DEoAn19LMcMs3b6ctdNF8K2hYd6d+BSZ
=TWit
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Jan 13 18:24:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jan 2010 12:24:42 -0500
Subject: [Python-Dev] PYTHON3PATH
Message-ID: <20100113172442.4A7771FA679@kimball.webabinitio.net>

Please review issue 2375 [1], which is an enhancement request to add a
PYTHON3PATH environment variable.  Because we have elected to have both
a python and a python3 command, I think this is an issue worth thinking
about carefully to make sure we are serving the Python user community
and easing the transition to python3.  It could be that "use virtualenv"
is the best answer, but I feel we should think about it carefully to
make sure that is really true.

[1] http://bugs.python.org/issue2375

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From guido at python.org  Wed Jan 13 18:31:39 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:31:39 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>

Somehow the bug site doesn't load for me right now, but I'm -1 on
this. There are maybe a dozen PYTHON... variables -- we really
shouldn't try to add PYTHON3 variants for all of them.

Specifically for PYTHONPATH, I feel that its use is always a
short-time or localized hack, not something you set in your .profile
nd forget about it (forgetting about it inparticular often leads to
unnecessary debugging pain later). The better approaches are based on
site-packages, wrapper scripts, virtualenv, etc.

--Guido

On Wed, Jan 13, 2010 at 9:24 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable. ?Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3. ?It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375
>
> --
> R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From dirkjan at ochtman.nl  Wed Jan 13 18:38:26 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 13 Jan 2010 18:38:26 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <ea2499da1001130938v68827914yc41476c75b5f3c62@mail.gmail.com>

On Sat, Jan 9, 2010 at 18:29, Benjamin Peterson <benjamin at python.org> wrote:
> Please consider trying Python 2.7 with your code and reporting any bugs you may

I ran the Mercurial test suite on current trunk, and it worked
alright. There was a small issue with the fact that mimetypes got
fooled by the lack of ImportError from _winreg (which I solved by
blacklisting _winreg in demandimport), and one test fails likely
because of a change in MIME handling. Will look into that. So the
state of trunk looks rather solid from where I sit.

Cheers,

Dirkjan


From ralf at brainbot.com  Wed Jan 13 18:40:04 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:40:04 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net> (R. David
	Murray's message of "Wed, 13 Jan 2010 12:24:42 -0500")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <87r5ptdhu3.fsf@muni.brainbot.com>

"R. David Murray" <rdmurray at bitdance.com> writes:

> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable.  Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3.  It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375

The first thing I got while trying to run a python3 prompt few days ago,
was an error. python3 tried to read my $PYTHONSTARTUP file, which used
print statements. people will have to run both python 2 and python 3
code at the same time. Using different environment variables will make
this easier.

- Ralf


From ralf at brainbot.com  Wed Jan 13 18:55:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:55:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
	(Guido van Rossum's message of "Wed, 13 Jan 2010 09:31:39 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
Message-ID: <87k4vldh4c.fsf@muni.brainbot.com>

Guido van Rossum <guido at python.org> writes:

> Somehow the bug site doesn't load for me right now, but I'm -1 on
> this. There are maybe a dozen PYTHON... variables -- we really
> shouldn't try to add PYTHON3 variants for all of them.
>
> Specifically for PYTHONPATH, I feel that its use is always a
> short-time or localized hack, not something you set in your .profile
> nd forget about it (forgetting about it inparticular often leads to
> unnecessary debugging pain later). The better approaches are based on
> site-packages, wrapper scripts, virtualenv, etc.

then at least remove them completely from python3, so one can use them
for his python 2 installation. Otherwise it will stump on some innocent
python 3 program (and cause unnecessary debugging pain).



From steven.bethard at gmail.com  Wed Jan 13 18:57:29 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 13 Jan 2010 09:57:29 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:40 AM, Ralf Schmitt <ralf at brainbot.com> wrote:
> "R. David Murray" <rdmurray at bitdance.com> writes:
>
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable. ?Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3. ?It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>>
>> [1] http://bugs.python.org/issue2375
>
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

How complicated is your PYTHONSTARTUP file? My suspicion is that you
could easily write it to work for both Python 2.X and 3.X.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From ssteinerx at gmail.com  Wed Jan 13 18:57:42 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 13 Jan 2010 12:57:42 -0500
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>


On Jan 13, 2010, at 12:40 PM, Ralf Schmitt wrote:

> "R. David Murray" <rdmurray at bitdance.com> writes:
> 
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable.  Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3.  It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>> 
>> [1] http://bugs.python.org/issue2375
> 
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

Or, how about just removing the antiquated use of environment variables altogether from Python 3 and avoid the issue completely.  

Python 2 will continue to run as it has, and better solutions will emerge for Python 3 development.

Beats the heck out of making a whole duplicate set for PYTHON3BLAHBLAH, yuck!

S



From guido at python.org  Wed Jan 13 18:58:04 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:58:04 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
	<bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
Message-ID: <ca471dc21001130958k6edabaacof256b5e811c75ea6@mail.gmail.com>

Memories of days past... Python had several regular expression
implementations before, one of which was called "regex".

But I would rather not have a new module -- I would much rather have a
flag specifying the new (backwards incompatible) syntax/semantics. The
flag would have a long name (e.g. re.NEW_SYNTAX), a short name (e.g.
re.N) and an inline syntax, "(?n)...".

--Guido

On Tue, Jan 12, 2010 at 7:58 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
>>
>
> If it is a separate module under a different name it can do the proper
> thing. People will just need to be aware of the difference when they import
> the module.
>
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
>>
>
> That's totally up to you. There is practically no chance of it getting into
> the 2.x under the stdlib at this point since 2.7b1 is coming up and this
> module has not been out in the wild for a year (to my knowledge). ?If you
> want to support 2.x that's fine and I am sure users would appreciate it, but
> it isn't necessary to get into the Python 3 stdlib.
>
>>
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
>>
>
> If you want to minimize the differences then it should probably match. As I
> said, since it is a different name to import under it can deviate where
> reasonable, just make sure to clearly document the deviations.
> -Brett
>
>>
>> Thanks
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)


From ralf at brainbot.com  Wed Jan 13 19:03:08 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 19:03:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
	(Steven Bethard's message of "Wed, 13 Jan 2010 09:57:29 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
Message-ID: <87fx69dgrn.fsf@muni.brainbot.com>

Steven Bethard <steven.bethard at gmail.com> writes:

>
> How complicated is your PYTHONSTARTUP file? My suspicion is that you
> could easily write it to work for both Python 2.X and 3.X.

sure. that's exactly what I did. My point is that sharing those
environment variables will cause pain for some people.


From guido at python.org  Wed Jan 13 19:07:58 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:07:58 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
just removing the antiquated use of environment variables altogether
from Python 3 and avoid the issue completely.

-1. They have their use, but more in controlled situations. If you
have "global" env vars that you only want to use with Python 2.x,
write a wrapper for Python 3 that invokes it with -E.

-- 
--Guido van Rossum (python.org/~guido)


From scott+python-dev at scottdial.com  Wed Jan 13 19:14:21 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 13 Jan 2010 13:14:21 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E0D7D.3040806@scottdial.com>

On 1/13/2010 9:04 AM, Nick Coghlan wrote:
> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

That is not correct. There are IA-64 versions of Window Server.

>From [1]:
"Backward compatibility is a key point differentiating Itanium from x86
and x64 architectures."

So to echo what Michael said, the Microsoft nomenclature is "x64"
regardless of yours and Martin's objections to that name. Nobody who
uses Windows would be confused by "x64" since that is *the* Microsoft
naming scheme. And, anyone using IA64 will expect to see "IA64" and not
"x64", and furthermore anyone using IA64 is likely aware of what
processor they are using (being as there are only Windows Server
editions for IA64) and the difference between "IA64" and "x64".

I don't think an end-user facing page is a great place for pedantic and
wordy defenses of defying the Microsoft-blessed naming scheme.

> Linux, on the other hand, can run on multiple 64 bit architectures, so
> being more specific and using the official AMD64 nomenclature makes
> sense.

This has no relevance to the conversation since there are no Linux
binaries being distributed. The conversation on the expectations of
Windows end-users, who are the target of the download links.

[1] http://www.microsoft.com/servers/64bit/itanium/overview.mspx

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From guido at python.org  Wed Jan 13 19:22:33 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:22:33 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <loom.20100113T131816-515@post.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<loom.20100113T131816-515@post.gmane.org>
Message-ID: <ca471dc21001131022w429e4ceal899235bc711ecaca@mail.gmail.com>

On Wed, Jan 13, 2010 at 4:24 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> If there is something missing from the stdlib discussion that you think should
> be brought up at the summit let me know. And if there is something here you want
> to discuss before the summit let me know and I can start a separate thread on it.
>
> I'm sorry I won't be able to come to the language summit, but I would like if
> possible to expedite a pronouncement on PEP 391 (configuring logging using
> dictionaries). I believe I addressed all the comments made on the discussion
> threads mentioned in the PEP and so I'm not sure what more I need to do to get a
> pronouncement. I guess the stdlib slot gives an opportunity for people to air
> their views and so I'd be grateful if you added it to the agenda.

Is the PEP in the stage of consensus yet? If so it may not even be
necessary to discuss it at the summit (though I certainly won't stop
people if they want to).

-- 
--Guido van Rossum (python.org/~guido)


From phd at phd.pp.ru  Wed Jan 13 19:18:50 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 21:18:50 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <20100113181850.GA3837@phd.pp.ru>

On Wed, Jan 13, 2010 at 12:57:42PM -0500, ssteinerX at gmail.com wrote:
> Or, how about just removing the antiquated use of environment variables

   "antiquated"? You are kidding, aren't you?!

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From guido at python.org  Wed Jan 13 19:51:14 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:51:14 -0800
Subject: [Python-Dev] PyCon Keynote
Message-ID: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>

Please mail me topics you'd like to hear me talk about in my keynote
at PyCon this year.

-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Wed Jan 13 20:13:58 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:13:58 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D9310.6060907@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de> <4B4D9310.6060907@voidspace.org.uk>
Message-ID: <4B4E1B76.4010309@v.loewis.de>

>>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>> X86-64 binary -- does not include source)
>>>
>>> instead of:
>>>
>>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>> not include source)
>>>      
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
>>    
> 
> I think we should use whatever is most informative and least confusing
> to our users - we owe our allegiance to them and not to a processor vendor.

And why do you think this is x86-64?

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:33:24 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:33:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA0DA.7070007@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk> <4B4DA0DA.7070007@gmail.com>
Message-ID: <4B4E2004.8050905@v.loewis.de>

> Note that increased 3.x compatibility in the most recent 2.x release
> will always help in two scenarios:
> 
> 1. New projects that want to use 2.x only libraries but want to be ready
> for the Py3k transition in their own code (e.g. new 2.7 features like
> set literals, dict and set comprehensions and the view* dict methods can
> be translated far more cleanly by 2to3 than the closest comparable 2.6
> code).

Of these, I can only see this being the case of the view* methods.

Why would set literals allow for code that more cleanly translates
to 3.x? Suppose you write

 f = set((1,2,3))

in 2.6. That code will work *exactly* the same way in 3.1, no
translation needed at all. Likewise for dict and set comprehensions:
the code is as cleanly translated in the 2.7 form as it is in any
prior form (ie. no modifications).

As for view* methods: there is one important exception where
the 2.6 equivalent is as cleanly translated as a view method:

for key in D:
  action

This is a more modern equivalent for iterating over D.keys(),
so if your code still does the latter, just change it to the
former (requires Python 2.2). I'd claim that this is the dominant
case of traversing a dictionary (probably followed by iterating
over .items()). So while it is true that only view* can be
transformed correctly, I'd claim that this is an infrequent
issue.

> 2. Similarly new projects that use a 3.x trunk can be more readily
> translated to a 2.7 version with 3to2, whereas some constructs may be
> difficult to recreate in earlier Python versions.

That may be true, but is besides the point here: the issue was whether
a 2.8 release might help in migrating to 3.x. People who follow this
approach already have 3.x code. Whether they would rather port to 2.8
only, and get that work with little effort, or whether they would port
to 2.5 and later, and put in more work, I don't know - I guess we would
have to ask these people. I would expect that they prefer if 2.x
died rather sooner than later, because they can then stop supporting
it.

Regards,
Martin



From tjreedy at udel.edu  Wed Jan 13 20:36:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 Jan 2010 14:36:03 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <hil7av$52r$1@ger.gmane.org>

On 1/13/2010 2:13 PM, "Martin v. L?wis" wrote:

>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>
> And why do you think this is x86-64?

I do not believe I have personally seen, or at least noticed, that 
specific term before. Something like

"Release for 64-bit Windows (Win NT, 2000, XP, Vista, 7 <or whatever 
correct list is>) on AMD64-type processors from AMD and Intel"

should be clear enough.




From martin at v.loewis.de  Wed Jan 13 20:40:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 20:40:30 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E21AE.40602@v.loewis.de>

> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Windows actually does - it runs on IA-64 (which is a non-x86 architecture).

However, end users are unlikely to use such hardware, so distinguishing
between 32-bit and 64-bit is typically good enough.

> In this case, making it clear that the 64-bit Windows binaries work for
> both Intel and AMD chips is more important than using only the official
> architecture name.

Well, we did have Itanium binaries at one point, and the "64-bit Windows
binaries" you are referring to most definitely don't run on Itanium,
even though that's an Intel chip.

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:45:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:45:40 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E0D7D.3040806@scottdial.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>	<4B4DD2EC.3030709@gmail.com>
	<4B4E0D7D.3040806@scottdial.com>
Message-ID: <4B4E22E4.4040504@v.loewis.de>

> So to echo what Michael said, the Microsoft nomenclature is "x64"
> regardless of yours and Martin's objections to that name. Nobody who
> uses Windows would be confused by "x64" since that is *the* Microsoft
> naming scheme.

That's actually not entirely true. There are several places in the
APIs where Microsoft either allows or requires to call the architecture
AMD64 (e.g. architecture indication in MSI files).

Regards,
Martin


From regebro at gmail.com  Wed Jan 13 20:50:59 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 20:50:59 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>

On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

What do you need to do in the PYTHONSTARTUP file?
Ten years of Python programming, and I didn't even know it existed. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From phd at phd.pp.ru  Wed Jan 13 21:08:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 23:08:40 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <20100113200840.GC14858@phd.pp.ru>

On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From murman at gmail.com  Wed Jan 13 21:12:11 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 14:12:11 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E22E4.4040504@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com> <4B4E0D7D.3040806@scottdial.com>
	<4B4E22E4.4040504@v.loewis.de>
Message-ID: <dcbbbb411001131212q3ef907c8i67e8c03c96c31e2f@mail.gmail.com>

On Wed, Jan 13, 2010 at 13:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So to echo what Michael said, the Microsoft nomenclature is "x64"
>> regardless of yours and Martin's objections to that name. Nobody who
>> uses Windows would be confused by "x64" since that is *the* Microsoft
>> naming scheme.
>
> That's actually not entirely true. There are several places in the
> APIs where Microsoft either allows or requires to call the architecture
> AMD64 (e.g. architecture indication in MSI files).

I should have clarified I was talking about the names shown on MSDN
subscriptions for downloading installation media of Windows 7 and
Windows Vista. It's pretty clear in that context Microsoft uses x64 to
describe this platform of Windows. But again, it's far from clear that
this is a term they use for non-developers.

-- 
Michael Urman


From regebro at gmail.com  Wed Jan 13 21:27:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 21:27:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113200840.GC14858@phd.pp.ru>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<20100113200840.GC14858@phd.pp.ru>
Message-ID: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>

On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>> What do you need to do in the PYTHONSTARTUP file?
>> Ten years of Python programming, and I didn't even know it existed. :-)
>
> ? See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Cheese and fries! :-)

Anyway, I don't see how this could pose any problems, because any user
advanced enough to do these things will be advanced enough to
understand what the problem is and fix it so it works under Python 3
too.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ralf at brainbot.com  Wed Jan 13 21:52:34 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 20:52:34 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	(Lennart Regebro's message of "Wed, 13 Jan 2010 20:50:59 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <874ompg225.fsf@brainbot.com>

Lennart Regebro <regebro at gmail.com> writes:

> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
>
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

hehe. tab completion:

# -*- mode: python -*- 
# Last changed: 2009-12-23 22:25:15 by ralf

import sys
import os

def initreadline():
    
    try:
        import readline
    except ImportError:
        sys.stdout.write("Module readline not available.\n")
        return
    
    import rlcompleter
    readline.parse_and_bind("tab: complete")
    
    # Use tab for completions
    readline.parse_and_bind('tab: complete')
    # This forces readline to automatically print the above list when tab
    # completion is set to 'complete'.
    readline.parse_and_bind('set show-all-if-ambiguous on')
    # Bindings for incremental searches in the history. These searches
    # use the string typed so far on the command line and search
    # anything in the previous input history containing them.
    readline.parse_and_bind('"\C-r": reverse-search-history')
    readline.parse_and_bind('"\C-s": forward-search-history')

    history = os.path.expanduser("~/.pyhistory") 
    if os.path.exists(history):
        readline.read_history_file(history)
        
    import atexit
    atexit.register(lambda: readline.write_history_file(history))
    
initreadline()
del initreadline


From martin at v.loewis.de  Wed Jan 13 22:05:03 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 22:05:03 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <4B4E357F.4050107@v.loewis.de>

Lennart Regebro wrote:
> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
> 
> What do you need to do in the PYTHONSTARTUP file?

I personally set up readline: set tab completion, and load the history
of commands from the previous session.

Of course, I don't know what Ralf uses it for.

Regards,
Martin


From ncoghlan at gmail.com  Wed Jan 13 22:43:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 07:43:41 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
Message-ID: <4B4E3E8D.2010407@gmail.com>

Guido van Rossum wrote:
> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
> just removing the antiquated use of environment variables altogether
> from Python 3 and avoid the issue completely.
> 
> -1. They have their use, but more in controlled situations. If you
> have "global" env vars that you only want to use with Python 2.x,
> write a wrapper for Python 3 that invokes it with -E.

Perhaps a case can be made for Python 3 to assume -E by default, with a
-e option to enable reading of the environment variables?

That way naive users could run Python3 without tripping over existing
Python2 environment variables, while other tools could readily set up a
different environment before launching Python3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From guido at python.org  Wed Jan 13 23:45:57 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 14:45:57 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com> 
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com> 
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <ca471dc21001131445g639c6867ude83f03a77eb72b9@mail.gmail.com>

On Wed, Jan 13, 2010 at 1:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
>
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
>
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users using environment variables? That's a recipe for disaster
in any version! :-)

-- 
--Guido van Rossum (python.org/~guido)


From jared.grubb at gmail.com  Wed Jan 13 23:56:37 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Wed, 13 Jan 2010 14:56:37 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <13F6C43A-D62A-4A9F-AF7A-9BDAC94F7D72@gmail.com>

On 13 Jan 2010, at 13:43, Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>> 
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

I raised a question about these PYTHON3* variables once before in a discussion about shebang lines:
http://www.mailinglistarchive.com/python-dev at python.org/msg29500.html

I'm not advocating them, but just wanted to make sure to bring up the shebang use case.

Jared

From skip at pobox.com  Wed Jan 13 23:24:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 16:24:13 -0600
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <19278.18445.428716.321365@montanaro.dyndns.org>


    Lennart> What do you need to do in the PYTHONSTARTUP file?

Just reading off stuff from my own personal startup file...  I use it for
stuff I want available during interactive sessions:

    1. Enable true division.

    2. Conditionally define "help" from back in the days when there was no
       help builtin function.

    3. Auto-save session (readline) history so I can easily recall commands
       across sessions.

    4. Add other fake builtins ("like") or override behavior of some (like
       "dir") making them handier for interactive use.

    5. autoload modules/symbols (pokes around in common modules from
       sys.excepthook function).

Oh, and I've had no particular trouble keeping it working in Python 1, 2 or
3.

Skip




From fuzzyman at voidspace.org.uk  Thu Jan 14 01:20:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 00:20:21 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <4B4E6345.5070202@voidspace.org.uk>

On 13/01/2010 19:13, "Martin v. L?wis" wrote:
>>>>    * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>>> X86-64 binary -- does not include source)
>>>>
>>>> instead of:
>>>>
>>>>    * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>>> not include source)
>>>>
>>>>          
>>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>>> to use AMD64 instead. I think we should comply - they invented the
>>> architecture, so they have the right to give it a name. Neither
>>> Microsoft nor Intel have such a right.
>>>
>>>        
>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>>      
> And why do you think this is x86-64?
>    

Well anecdotal everyone I have *every* talked to about 64bit processors 
has referred to having a 64bit processor (x86 is a given) and not an 
AMD64 architecture processor.

Linus Torvalds addressed this specific issue for Linux and came down on 
the side of "x86-64": http://kerneltrap.org/node/2466
Look up AMD64 on Wikipedia and it redirects you to the X86-64 page.
Information website setup by AMD and partners about the AMD64 
architecture: http://www.x86-64.org/about.html
In the AMD website they refer to "x86-64 Assembly": 
http://www.x86-64.org/documentation/assembly.html

Microsoft seem to draw a distinction between x64 (which would also be 
acceptable) and Itanium based systems. Very rarely do they refer to AMD64:

* http://www.microsoft.com/servers/64bit/compare.mspx
* http://www.microsoft.com/servers/64bit/x64/overview.mspx
* http://www.microsoft.com/servers/64bit/overview.mspx

Using a vendor specific name automatically begs the question as to 
whether the installer works on processors from other vendors, as we saw 
in the specific enquiry from the user that triggered this debate.

Referring to the AMD 64 build as x86-64, with a footnote explaining 
which architectures this specifically means is unlikely to confuse 
people. It is *definitely* better than just saying AMD64.

All the best,

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From skip at pobox.com  Thu Jan 14 02:50:55 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 19:50:55 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
Message-ID: <19278.30847.649228.115514@montanaro.dyndns.org>


Just out of curiosity, is the static build stuff (use the old Modules/Setup
file to build modules) exercised at all any more?

Skip



From benjamin at python.org  Thu Jan 14 03:22:03 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 13 Jan 2010 20:22:03 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
In-Reply-To: <19278.30847.649228.115514@montanaro.dyndns.org>
References: <19278.30847.649228.115514@montanaro.dyndns.org>
Message-ID: <1afaf6161001131822r151bd202x35653ba44ee0235d@mail.gmail.com>

2010/1/13  <skip at pobox.com>:
>
> Just out of curiosity, is the static build stuff (use the old Modules/Setup
> file to build modules) exercised at all any more?

Exercised as in used in the build process? Yes.


-- 
Regards,
Benjamin


From vinay_sajip at yahoo.co.uk  Thu Jan 14 10:23:47 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 09:23:47 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
Message-ID: <954450.26617.qm@web25804.mail.ukl.yahoo.com>

In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:

  http://www.python.org/dev/peps/pep-0391/

In November 2009 I posted to this list that the PEP was ready for review.

I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.

So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.

Thanks & regards,

Vinay Sajip



      



From ziade.tarek at gmail.com  Thu Jan 14 10:30:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 Jan 2010 10:30:15 +0100
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <94bdd2611001140130u6154e893jfd37db32a25be66a@mail.gmail.com>

On Thu, Jan 14, 2010 at 10:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[..]
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would
> be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check
> in the changes unless there are objections to doing so. All those who think that logging is currently
> hard to configure will benefit from these changes, so here's the opportunity to pipe up and
> improve things.


FWIW, I am +1. Thanks for your work

Tarek


From hodgestar+pythondev at gmail.com  Thu Jan 14 10:39:41 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 14 Jan 2010 11:39:41 +0200
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <fb73205e1001140139n4b920871t6bfc6b91c469264f@mail.gmail.com>

On Thu, Jan 14, 2010 at 11:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I think the dict configuration scheme will be a great addition to the
logging module. :)

Schiavo
Simon


From p.f.moore at gmail.com  Thu Jan 14 12:22:09 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 Jan 2010 11:22:09 +0000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>

2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I've no immediate need for the feature, but it would be good to have
something like this, so I'm +1.
Paul.


From ncoghlan at gmail.com  Thu Jan 14 12:46:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:46:19 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
Message-ID: <4B4F040B.8010607@gmail.com>

Paul Moore wrote:
> 2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> So, can you please indicate your vote for or against incorporating PEP 391 into Python?
> 
> I've no immediate need for the feature, but it would be good to have
> something like this, so I'm +1.

I'm in the same boat as Paul, but PEP 291 provides a nice forward
compatible configuration scheme that will work with any application
configuration approach that can produce an appropriate Python dictionary.

So +1 from me too - I think the PEP has now taken this as far as
theorising can go, and the only way to learn anything further is to put
it into practice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Jan 14 12:57:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:57:55 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4B4F06C3.7030806@gmail.com>

Vinay Sajip wrote:
> In October 2009 I created PEP 391 to propose a new method of
> configuring logging using dictionaries:
> 
> http://www.python.org/dev/peps/pep-0391/

Although one minor comment: you can probably remove the note about the
"ext://" and "cfg://" prefixes being provisional at this stage. Those
names look fine to me, so I don't see any point inviting a late-breaking
bikeshed discussion about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Thu Jan 14 14:53:29 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 08:53:29 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>

On Thu, Jan 14, 2010 at 4:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:
>
> ?http://www.python.org/dev/peps/pep-0391/
>
> In November 2009 I posted to this list that the PEP was ready for review.
>
> I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.
>
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.
>
> Thanks & regards,
>
> Vinay Sajip

I'm generally +1 - but given I know that Django 1.2 is slated to
implement something somewhat similar, I'm interested to hear how this
proposal meshes with their plan(s).

jesse


From vinay_sajip at yahoo.co.uk  Thu Jan 14 15:08:24 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 14:08:24 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
Message-ID: <92210.91656.qm@web25806.mail.ukl.yahoo.com>

> From: Jesse Noller <jnoller at gmail.com>

> I'm generally +1 - but given I know that Django 1.2 is slated to
> implement something somewhat similar, I'm interested to hear how this
> proposal meshes with their plan(s)..

Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:

http://groups.google.com/group/django-developers/msg/4ef81a2160257221

They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).

Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

Regards,

Vinay Sajip


      



From ssteinerx at gmail.com  Thu Jan 14 15:54:27 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 14 Jan 2010 09:54:27 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <62E362ED-5DD7-4D42-B5E0-B263A137FD5C@gmail.com>


On Jan 14, 2010, at 9:08 AM, Vinay Sajip wrote:

>> From: Jesse Noller <jnoller at gmail.com>
> 
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
> 
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
> 
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
> 
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
> 
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

That puts a huge +1 on there for me.  

If we can get this approved and have Django's logging improved *and* avoid a whole pile of duplicate effort in Django that's a huge win.

S



From jnoller at gmail.com  Thu Jan 14 15:56:18 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 09:56:18 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140656w68c7290agccfe7282cf96f2fb@mail.gmail.com>

On Thu, Jan 14, 2010 at 9:08 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> From: Jesse Noller <jnoller at gmail.com>
>
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
>
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
>
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
>
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
>
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.
>
> Regards,
>
> Vinay Sajip
>

Cool, +1 then :)


From mal at egenix.com  Thu Jan 14 20:19:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 20:19:12 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <4B4F6E30.50400@egenix.com>

Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users won't have PYTHONPATH or any other Python environment
variables setup.

Really, if you know that you are going to run Python 3 instead of
Python 2 or vice-versa it's easy enough to run

. py3env.sh
or
. py2env.sh

in order to setup your shell environment, much like you typically
do when using virtual Python installations or work on different
projects that require different setups.

If you just want to separate Python 2 and 3 files, use the
user site-packages dir which includes the Python version.

More experienced users could also write their own environment
switching sitecustomize.py or usercustomize.py.

And then there's the hackery option for experts that love
dirty tricks: add a .pth file which includes an "import mysetup"
line to fix-up you path and other settings.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Jan 14 22:02:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 15 Jan 2010 07:02:28 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <4B4F8664.4080107@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Guido van Rossum wrote:
>>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>>> just removing the antiquated use of environment variables altogether
>>> from Python 3 and avoid the issue completely.
>>>
>>> -1. They have their use, but more in controlled situations. If you
>>> have "global" env vars that you only want to use with Python 2.x,
>>> write a wrapper for Python 3 that invokes it with -E.
>> Perhaps a case can be made for Python 3 to assume -E by default, with a
>> -e option to enable reading of the environment variables?
>>
>> That way naive users could run Python3 without tripping over existing
>> Python2 environment variables, while other tools could readily set up a
>> different environment before launching Python3.
> 
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.

I was actually thinking of the situation where the OS had a preinstalled
Python 2 with a default PYTHONPATH setting and the user was playing with
Python 3.

However, I agree that that is a fairly unlikely scenario (since
preinstalled Pythons tend not to rely on the environment variables and a
user sophisticated enough to be playing with a new Python interpreter
should be able to cope with a few environment variable conflicts anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at voidspace.org.uk  Thu Jan 14 22:09:30 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 21:09:30 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F8664.4080107@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <4B4F8664.4080107@gmail.com>
Message-ID: <4B4F880A.5010809@voidspace.org.uk>

On 14/01/2010 21:02, Nick Coghlan wrote:
> However, I agree that that is a fairly unlikely scenario (since
> preinstalled Pythons tend not to rely on the e
Well, on the other hand I think that during the next few years it will 
be increasingly common for developers (and possibly users) to have 
Python 2 and Python 3 installed side-by-side.

Many libraries and applications may never make the jump to Python 3 and 
Python users may be using 'legacy' Python 2 code for many years to come. 
It will also become increasingly common for developers to be using 
Python 3 *primarily* and for Python 3 only libraries and applications to 
emerge.

Whilst there are workarounds we *are* in a situation that Python 2 and 
Python 3 share environment variables for the location of libraries and 
executing code on startup, whilst at the same time they are largely 
incompatible and need separate library paths and startup code.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ralf at brainbot.com  Thu Jan 14 22:25:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 14 Jan 2010 22:25:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com> (M.'s message of "Thu, 14 Jan 2010
	20:19:12 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <871vhswf90.fsf@brainbot.com>

"M.-A. Lemburg" <mal at egenix.com> writes:

>
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.
>
> Really, if you know that you are going to run Python 3 instead of
> Python 2 or vice-versa it's easy enough to run

You don't even know that you're going to run python. I have 40 python
scripts in my /usr/bin directory. 

>
> . py3env.sh
> or
> . py2env.sh
>
> in order to setup your shell environment, much like you typically
> do when using virtual Python installations or work on different
> projects that require different setups.

No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.

>
> If you just want to separate Python 2 and 3 files, use the
> user site-packages dir which includes the Python version.

Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
install those programs in a user's home directory. There are still
people running python <2.6.


From mal at egenix.com  Thu Jan 14 22:51:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 22:51:07 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <871vhswf90.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <871vhswf90.fsf@brainbot.com>
Message-ID: <4B4F91CB.2060106@egenix.com>

Ralf Schmitt wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>
>> Naive users won't have PYTHONPATH or any other Python environment
>> variables setup.
>>
>> Really, if you know that you are going to run Python 3 instead of
>> Python 2 or vice-versa it's easy enough to run
> 
> You don't even know that you're going to run python. I have 40 python
> scripts in my /usr/bin directory. 
> 
>>
>> . py3env.sh
>> or
>> . py2env.sh
>>
>> in order to setup your shell environment, much like you typically
>> do when using virtual Python installations or work on different
>> projects that require different setups.
> 
> No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.
> 
>>
>> If you just want to separate Python 2 and 3 files, use the
>> user site-packages dir which includes the Python version.
> 
> Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
> install those programs in a user's home directory. There are still
> people running python <2.6.

Note that you only need to have a different PYTHONPATH
setups for Python 2.x/3.x if you plan to run code that:

 * is not installed in site-packages (most OS shipped code
   will be found in the resp. system site-packages/ dir)

 * is not installed in a user site-packages directory
   (user installed code will typically go there (*))

 * uses modules/packages that come in two different versions
   (one for Python 2.x and one for 3.x) and use the same name
   for both versions

 * needs to work in both Python 2.x and 3.x


(*) The method for installing code in user site-packages dir is
running:

    python setup.py install --user

instead of the standard

    python setup.py install

which install to the system-wide site-packages.

BTW: I guess the bzr and mercurial wikis will need to be updated
accordingly - at least for users of Python >=2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From martin at v.loewis.de  Thu Jan 14 22:55:04 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Jan 2010 22:55:04 +0100
Subject: [Python-Dev] [ANN] Python 2.5.5 Release Candidate 1.
Message-ID: <4B4F92B8.30806@v.loewis.de>

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.5.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.6 (which is
2.6.4 at this point).

This releases fixes issues with the logging and tarfile modules, and
with thread-local variables. See the detailed release notes at the
website (also available as Misc/NEWS in the source distribution) for
details of bugs fixed.

For more information on Python 2.5.5, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.5.5

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

    http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
martin at v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)


From status at bugs.python.org  Fri Jan 15 18:07:26 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jan 2010 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100115170726.9963A7865F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/08/10 - 01/15/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2536 open (+32) / 16993 closed (+16) / 19529 total (+48)

Open issues with patches:  1024

Average duration of open issues: 710 days.
Median duration of open issues: 469 days.

Open Issues Breakdown
   open  2501 (+32)
pending    34 ( +0)

Issues Created Or Reopened (53)
_______________________________

PYTHON3PATH environment variable to supersede PYTHONPATH for mul 01/13/10
CLOSED http://bugs.python.org/issue2375    reopened r.david.murray                
                                                                               

Mark the compiler package as deprecated                          01/13/10
       http://bugs.python.org/issue6837    reopened ezio.melotti                  
                                                                               

test_distutils failure                                           01/15/10
CLOSED http://bugs.python.org/issue6961    reopened flox                          
       buildbot                                                                

test_urllib: unsetting missing 'env' variable                    01/08/10
CLOSED http://bugs.python.org/issue7026    reopened flox                          
                                                                               

Two float('nan') are not equal                                   01/08/10
CLOSED http://bugs.python.org/issue7660    created  jmfauth                       
                                                                               

compiling ctypes fails with non-ascii path                       01/08/10
       http://bugs.python.org/issue7661    created  pitrou                        
       patch                                                                   

time.utcoffset()                                                 01/09/10
       http://bugs.python.org/issue7662    created  techtonik                     
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points  01/10/10
CLOSED http://bugs.python.org/issue7663    created  exarkun                       
                                                                               

python logger does not handle IOError Exception                  01/10/10
CLOSED http://bugs.python.org/issue7664    created  yateenjoshi                   
                                                                               

test_urllib2 and test_ntpath fail if path contains "\"           01/10/10
       http://bugs.python.org/issue7665    created  flox                          
                                                                               

test_lib2to3 fails if path contains space                        01/10/10
CLOSED http://bugs.python.org/issue7666    created  flox                          
                                                                               

test_doctest fails with non-ascii path                           01/10/10
       http://bugs.python.org/issue7667    created  flox                          
       buildbot                                                                

test_httpservers fails with non-ascii path                       01/10/10
       http://bugs.python.org/issue7668    created  flox                          
       buildbot                                                                

test_unicode_file fails with non-ascii path                      01/10/10
CLOSED http://bugs.python.org/issue7669    created  flox                          
                                                                               

_sqlite3: Block *all* operations on a closed Connection object   01/10/10
       http://bugs.python.org/issue7670    created  haypo                         
       patch                                                                   

test_popen fails if path contains special char like ";"          01/11/10
       http://bugs.python.org/issue7671    reopened flox                          
       patch                                                                   

_ssl module causes segfault                                      01/10/10
       http://bugs.python.org/issue7672    created  ssoria                        
       patch                                                                   

audioop: check that length is a multiple of the size             01/11/10
       http://bugs.python.org/issue7673    created  haypo                         
       patch                                                                   

select.select() corner cases: duplicate fds, out-of-range fds    01/11/10
       http://bugs.python.org/issue7674    created  cdleary                       
                                                                               

help() doesn't accept unicode literals in built in docstrings    01/11/10
       http://bugs.python.org/issue7675    created  psd                           
       patch                                                                   

IDLE shell shouldn't use TABs                                    01/11/10
       http://bugs.python.org/issue7676    created  cben                          
                                                                               

distutils, better error message for setup.py upload -sign withou 01/11/10
       http://bugs.python.org/issue7677    created  illume                        
                                                                               

subprocess.Popen pipeline example code in the documentation is l 01/11/10
       http://bugs.python.org/issue7678    created  steven.k.wong                 
                                                                               

Warning building 2.7 on OS X 10.6 libintl.h "Present But Cannot  01/12/10
       http://bugs.python.org/issue7679    created  ssteinerX                     
                                                                               

pythonw crash while attempting to start() a thread object        01/12/10
       http://bugs.python.org/issue7680    created  dontbugme                     
                                                                               

Incorrect division in [wave.py]                                  01/12/10
CLOSED http://bugs.python.org/issue7681    created  alfps                         
       patch, needs review                                                     

Optimisation of if with constant expression                      01/12/10
       http://bugs.python.org/issue7682    created  sdefresne                     
       patch                                                                   

Wrong link in HTML documentation                                 01/12/10
CLOSED http://bugs.python.org/issue7683    created  francescor                    
                                                                               

decimal.py: infinity coefficients in tuples                      01/12/10
       http://bugs.python.org/issue7684    created  skrah                         
                                                                               

minor typo in re docs                                            01/12/10
CLOSED http://bugs.python.org/issue7685    created  mikejs                        
       patch                                                                   

redundant open modes 'rbb', 'wbb', 'abb' no longer work on Windo 01/13/10
       http://bugs.python.org/issue7686    created  ivank                         
                                                                               

Bluetooth support untested                                       01/13/10
       http://bugs.python.org/issue7687    created  pitrou                        
                                                                               

TypeError: __name__ must be set to a string object               01/13/10
       http://bugs.python.org/issue7688    created  frankmillman                  
                                                                               

Pickling of classes with a metaclass and copy_reg                01/13/10
       http://bugs.python.org/issue7689    created  craigcitro                    
       patch                                                                   

Wrong PEP number in importlib module manual page                 01/13/10
CLOSED http://bugs.python.org/issue7690    created  francescor                    
                                                                               

test_bsddb3 files are not always removed when test fails         01/13/10
CLOSED http://bugs.python.org/issue7691    created  flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero              01/13/10
CLOSED http://bugs.python.org/issue7692    created  Bugger                        
                                                                               

tarfile.extractall can't have unicode extraction path            01/13/10
       http://bugs.python.org/issue7693    created  pbienst                       
                                                                               

DeprecationWarning from distuils.commands.build_ext needs stackl 01/13/10
       http://bugs.python.org/issue7694    created  tseaver                       
       patch                                                                   

missing termios constants                                        01/13/10
       http://bugs.python.org/issue7695    created  conorh                        
                                                                               

Improve Memoryview/Buffer documentation                          01/13/10
       http://bugs.python.org/issue7696    created  flox                          
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes  01/13/10
CLOSED http://bugs.python.org/issue7697    created  flox                          
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame  01/14/10
CLOSED http://bugs.python.org/issue7698    created  exarkun                       
       patch                                                                   

strptime, strftime documentation                                 01/14/10
       http://bugs.python.org/issue7699    created  mikejs                        
       patch                                                                   

"-3" flag does not work anymore                                  01/14/10
CLOSED http://bugs.python.org/issue7700    created  flox                          
       patch                                                                   

fix output string length for binascii.b2a_uu()                   01/14/10
CLOSED http://bugs.python.org/issue7701    created  haypo                         
       patch                                                                   

Wrong order of parameters of _get_socket in SMTP class in smtpli 01/14/10
       http://bugs.python.org/issue7702    created  alito                         
                                                                               

Replace buffer()-->memoryview() in Lib/ctypes/test/              01/14/10
       http://bugs.python.org/issue7703    created  flox                          
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE         01/14/10
CLOSED http://bugs.python.org/issue7704    created  DhaReaL                       
                                                                               

libpython2.6.so is not linked correctly on FreeBSD when threads  01/15/10
       http://bugs.python.org/issue7705    created  aballier                      
       patch, needs review                                                     

Missing #include guards                                          01/15/10
       http://bugs.python.org/issue7706    created  akrpic77                      
       patch                                                                   

multiprocess.Queue operations during import can lead to deadlock 01/15/10
       http://bugs.python.org/issue7707    created  kripken                       
                                                                               

Conversion of longs to bytes and vice-versa.                     01/11/10
       http://bugs.python.org/issue1023290 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (67)
______________________

segfault in curses when calling redrawwin() before refresh()      825 days
       http://bugs.python.org/issue1266    mark.dickinson                
                                                                               

"RuntimeError: dictionary changed size during iteration" in Tkin  751 days
       http://bugs.python.org/issue1658    flox                          
       patch                                                                   

Exception exceptions.AttributeError '_shutdown' in <module 'thre  718 days
       http://bugs.python.org/issue1947    nadiasvertex                  
                                                                               

Backport dictviews to 2.7                                         713 days
       http://bugs.python.org/issue1967    alexandre.vassalotti          
       patch, needs review                                                     

Backport set and dict comprehensions                              665 days
       http://bugs.python.org/issue2333    alexandre.vassalotti          
       patch, 26backport                                                       

Backport set literals                                             663 days
       http://bugs.python.org/issue2335    alexandre.vassalotti          
       patch, 26backport                                                       

PYTHON3PATH environment variable to supersede PYTHONPATH for mul    1 days
       http://bugs.python.org/issue2375    lemburg                       
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     625 days
       http://bugs.python.org/issue2698    cmcqueen1975                  
                                                                               

arg 2 of PyErr_SetFromErrnoWithFilename should be const           619 days
       http://bugs.python.org/issue2758    brian.curtin                  
                                                                               

Trunk build issues in DragonFly BSD                               613 days
       http://bugs.python.org/issue2797    brian.curtin                  
       patch, needs review                                                     

Gzip cannot handle zero-padded output + patch                     610 days
       http://bugs.python.org/issue2846    pitrou                        
       patch, needs review                                                     

block operation on closed socket/pipe for multiprocessing         552 days
       http://bugs.python.org/issue3311    haypo                         
       patch, needs review                                                     

Add gamma function, error functions and other C99 math.h functio  543 days
       http://bugs.python.org/issue3366    mark.dickinson                
       patch, needs review                                                     

Macros for PyLong: sign, number of digits, fits in an int         425 days
       http://bugs.python.org/issue4294    mark.dickinson                
       patch                                                                   

reject unicode in zlib                                            379 days
       http://bugs.python.org/issue4757    haypo                         
       patch                                                                   

Distutils inappropriately reuses .o files between extension modu  320 days
       http://bugs.python.org/issue5372    tarek                         
       patch, patch, needs review                                              

Problem with email.MIME* library, using wrong new line            298 days
       http://bugs.python.org/issue5525    r.david.murray                
                                                                               

os.path.normpath doesn't preserve unicode                         263 days
       http://bugs.python.org/issue5827    ezio.melotti                  
       patch                                                                   

smtplib exception smtp.connect TypeError encode_plain             173 days
       http://bugs.python.org/issue6523    r.david.murray                
                                                                               

email.parser clips trailing \n of multipart/mixed part if part e  152 days
       http://bugs.python.org/issue6681    r.david.murray                
       patch                                                                   

Optimize PyBytes_FromObject.                                      151 days
       http://bugs.python.org/issue6688    alexandre.vassalotti          
       patch                                                                   

in xmlrpclib.py: NameError: global name 'HTTPSConnection' is not  143 days
       http://bugs.python.org/issue6769    krisvale                      
                                                                               

UnicodeDecodeError when retrieving binary data from cgi.FieldSto  126 days
       http://bugs.python.org/issue6854    r.david.murray                
                                                                               

test_distutils failure                                              0 days
       http://bugs.python.org/issue6961    flox                          
       buildbot                                                                

tmpnam should not be used if tempfile or mkstemp are available    110 days
       http://bugs.python.org/issue6965    flox                          
                                                                               

test_urllib: unsetting missing 'env' variable                       0 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

distutils and IronPython compatibility                             73 days
       http://bugs.python.org/issue7071    tarek                         
       26backport                                                              

weak dict iterators are fragile because of unpredictable GC runs   89 days
       http://bugs.python.org/issue7105    pitrou                        
       patch                                                                   

email:  msg.items() returns different values before and after ms   89 days
       http://bugs.python.org/issue7119    r.david.murray                
       patch                                                                   

get_payload(decode=True) eats last newline                         87 days
       http://bugs.python.org/issue7143    r.david.murray                
                                                                               

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo   49 days
       http://bugs.python.org/issue7382    alexandre.vassalotti          
       patch, easy                                                             

Document inspect.get(full)argspec limitation to Python function    39 days
       http://bugs.python.org/issue7422    georg.brandl                  
                                                                               

Py3.1: Fatal Python Error: Py_Initialize...unknown encoding: chc   35 days
       http://bugs.python.org/issue7441    georg.brandl                  
                                                                               

when piping output between subprocesses some fd is left open blo   36 days
       http://bugs.python.org/issue7448    flox                          
                                                                               

Solaris SPARC: _multiprocessing.so: symbol sem_timedwait: refere   35 days
       http://bugs.python.org/issue7454    srid                          
                                                                               

cPickle: stack underflow in load_pop()                             33 days
       http://bugs.python.org/issue7455    haypo                         
       patch                                                                   

crash in str.rfind() with an invalid start value                   33 days
       http://bugs.python.org/issue7458    haypo                         
       patch                                                                   

test_multiprocessing test_rapid_restart fails if port 9999 alrea   27 days
       http://bugs.python.org/issue7498    r.david.murray                
       patch, buildbot                                                         

Extended slicing with classic class behaves strangely               3 days
       http://bugs.python.org/issue7532    mark.dickinson                
       patch                                                                   

stringlib fastsearch could be improved on 64-bit builds            13 days
       http://bugs.python.org/issue7607    pitrou                        
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option    7 days
       http://bugs.python.org/issue7617    tarek                         
       patch                                                                   

[patch] improve unicode methods: split() rsplit() and replace()    10 days
       http://bugs.python.org/issue7622    pitrou                        
       patch                                                                   

bytearray needs more tests for "b.some_method()[0] is not b"       10 days
       http://bugs.python.org/issue7625    pitrou                        
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                    4 days
       http://bugs.python.org/issue7648    orsenthil                     
       easy                                                                    

[patch] Enable additional bytes and memoryview tests.               5 days
       http://bugs.python.org/issue7654    pitrou                        
       patch                                                                   

test_ctypes failure on AIX 5.3                                      4 days
       http://bugs.python.org/issue7657    mallayya                      
       patch                                                                   

Two float('nan') are not equal                                      0 days
       http://bugs.python.org/issue7660    mark.dickinson                
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points     1 days
       http://bugs.python.org/issue7663    lemburg                       
                                                                               

python logger does not handle IOError Exception                     1 days
       http://bugs.python.org/issue7664    vinay.sajip                   
                                                                               

test_lib2to3 fails if path contains space                           0 days
       http://bugs.python.org/issue7666    benjamin.peterson             
                                                                               

test_unicode_file fails with non-ascii path                         2 days
       http://bugs.python.org/issue7669    ezio.melotti                  
                                                                               

Incorrect division in [wave.py]                                     1 days
       http://bugs.python.org/issue7681    benjamin.peterson             
       patch, needs review                                                     

Wrong link in HTML documentation                                    0 days
       http://bugs.python.org/issue7683    ezio.melotti                  
                                                                               

minor typo in re docs                                               0 days
       http://bugs.python.org/issue7685    ezio.melotti                  
       patch                                                                   

Wrong PEP number in importlib module manual page                    0 days
       http://bugs.python.org/issue7690    brett.cannon                  
                                                                               

test_bsddb3 files are not always removed when test fails            2 days
       http://bugs.python.org/issue7691    flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero                 0 days
       http://bugs.python.org/issue7692    mark.dickinson                
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes     0 days
       http://bugs.python.org/issue7697    pjenvey                       
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame     0 days
       http://bugs.python.org/issue7698    skip.montanaro                
       patch                                                                   

"-3" flag does not work anymore                                     1 days
       http://bugs.python.org/issue7700    brett.cannon                  
       patch                                                                   

fix output string length for binascii.b2a_uu()                      1 days
       http://bugs.python.org/issue7701    pitrou                        
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE            0 days
       http://bugs.python.org/issue7704    tim_one                       
                                                                               

Support for sending multipart form data                          2452 days
       http://bugs.python.org/issue727898  r.david.murray                
                                                                               

email.MIME*.as_string removes duplicate spaces                   1440 days
       http://bugs.python.org/issue1422094 r.david.murray                
       easy                                                                    

test_parsedate_acceptable_to_time_functions+DST == :-(           1394 days
       http://bugs.python.org/issue1454285 r.david.murray                
                                                                               

email module does not complay with RFC 2046: CRLF issue          1196 days
       http://bugs.python.org/issue1571841 r.david.murray                
                                                                               

email.FeedParser.BufferedSubFile improperly handles "\r\n"        969 days
       http://bugs.python.org/issue1721862 r.david.murray                
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 20 dtoa.c: oversize b in quorem                                      11 days
open    http://bugs.python.org/issue7632   

 18 _ssl module causes segfault                                        5 days
open    http://bugs.python.org/issue7672   

 12 Speed up cPickle's pickling generally                            287 days
open    http://bugs.python.org/issue5683   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    7 days
open    http://bugs.python.org/issue7658   

  8 Fatal error on thread creation in low memory condition            27 days
open    http://bugs.python.org/issue7544   

  8 Direct calls to PyObject_Del/PyObject_DEL are broken for --with  558 days
open    http://bugs.python.org/issue3299   

  7 "-3" flag does not work anymore                                    1 days
closed  http://bugs.python.org/issue7700   

  7 tarfile.extractall can't have unicode extraction path              2 days
open    http://bugs.python.org/issue7693   

  7 test_urllib: unsetting missing 'env' variable                      0 days
closed  http://bugs.python.org/issue7026   

  7 os.path.abspath with unicode argument should call os.getcwdu     542 days
open    http://bugs.python.org/issue3426   




From g.brandl at gmx.net  Sat Jan 16 21:05:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 16 Jan 2010 21:05:46 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<20100113200840.GC14858@phd.pp.ru>
	<319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
Message-ID: <hit67o$7v4$1@ger.gmane.org>

Am 13.01.2010 21:27, schrieb Lennart Regebro:
> On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
>> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>>> What do you need to do in the PYTHONSTARTUP file?
>>> Ten years of Python programming, and I didn't even know it existed. :-)
>>
>>   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.
> 
> Cheese and fries! :-)
> 
> Anyway, I don't see how this could pose any problems, because any user
> advanced enough to do these things will be advanced enough to
> understand what the problem is and fix it so it works under Python 3
> too.

I'd propose adding a bit of text to the environment variables documentation,
and especially about the needs of PYTHONSTARTUP files.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From asmodai at in-nomine.org  Sat Jan 16 21:50:56 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 16 Jan 2010 21:50:56 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <874ompg225.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
Message-ID: <20100116205056.GL99670@nexus.in-nomine.org>

-On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>hehe. tab completion:

With bpython and ipython available, why would you even want to stick to the
'plain old' interactive interpreter?

(Sorry to derail the discussion, but maybe there's more people that have not
heard of either or both.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For ever, brother, hail and farewell...


From solipsis at pitrou.net  Sat Jan 16 21:57:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jan 2010 20:57:13 +0000 (UTC)
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <loom.20100116T215639-769@post.gmane.org>

Jeroen Ruigrok van der Werven <asmodai <at> in-nomine.org> writes:
> 
> -On [20100113 22:13], Ralf Schmitt (ralf <at> brainbot.com) wrote:
> >hehe. tab completion:
> 
> With bpython and ipython available, why would you even want to stick to the
> 'plain old' interactive interpreter?

Why wouldn't we?
There are probably many more people using the standard Python prompt than
ipython.





From ben+python at benfinney.id.au  Sat Jan 16 23:41:03 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2010 09:41:03 +1100
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <87y6jxk70g.fsf@benfinney.id.au>

Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:

> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
> >hehe. tab completion:
>
> With bpython and ipython available, why would you even want to stick
> to the 'plain old' interactive interpreter?

Because those optional extras are not always available, whereas the
standard interactive interpreter is always available with CPython
distributions.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Sun Jan 17 01:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:22:10 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87y6jxk70g.fsf@benfinney.id.au>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<874ompg225.fsf@brainbot.com>	<20100116205056.GL99670@nexus.in-nomine.org>
	<87y6jxk70g.fsf@benfinney.id.au>
Message-ID: <4B525832.8090904@gmail.com>

Ben Finney wrote:
> Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:
> 
>> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>>> hehe. tab completion:
>> With bpython and ipython available, why would you even want to stick
>> to the 'plain old' interactive interpreter?
> 
> Because those optional extras are not always available, whereas the
> standard interactive interpreter is always available with CPython
> distributions.

I've never even contemplated the idea of installing 3rd party apps for
the 4 current development builds (2.x trunk, 2.x maintenance, 3.x trunk,
3.x maintenance) on my home development machine.

And of course work suffers from the problem of not being allowed to
install arbitrary apps I downloaded from the internet without getting
the licensing vetted for commercial acceptability.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 01:34:08 2010
From: nad at acm.org (Ned Deily)
Date: Sat, 16 Jan 2010 16:34:08 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
Message-ID: <nad-79FC16.16340816012010@news.gmane.org>

I've recently seen a couple of references to 3.1.2 go by in checkins 
which made me wonder whether dates have been proposed yet for updates to 
either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
references in the PEPs.  Some advance warning would be nice.  I assume 
that some critical problem could trigger planning for an update on short 
notice; is there a time-limit trigger as well?

-- 
 Ned Deily,
 nad at acm.org



From ncoghlan at gmail.com  Sun Jan 17 01:55:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:55:38 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52600A.7060201@gmail.com>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Usually there is one more regular maintenance release of the existing
maintenance branches within a few months of the creation of the next
version (releases before then are at the discretion of the release
manager, and security releases continue for much longer).

So take the planned 2.7 and 3.2 release dates and add a couple of months
to each one to get the likely release dates for the 2.6.x and 3.1.x
maintenance releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Sun Jan 17 10:21:49 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Jan 2010 10:21:49 +0100
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52D6AD.6090302@v.loewis.de>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Barry was once proposing time-based releases; this idea didn't really
catch on.

It's the release manager who decides when the next bug fix release will
be made, often in response to developers asking for one.

Regards,
Martin



From solipsis at pitrou.net  Sun Jan 17 14:00:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 13:00:04 +0000 (UTC)
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <loom.20100117T135910-313@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.

There are a couple of release blockers right now. Once they are fixed or
deferred, I think it would be nice to have a 3.1.2.
Why do you need "some advance warning" though?




From ncoghlan at gmail.com  Sun Jan 17 14:40:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 23:40:42 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <loom.20100117T135910-313@post.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
Message-ID: <4B53135A.7060104@gmail.com>

Antoine Pitrou wrote:
> Ned Deily <nad <at> acm.org> writes:
>> I've recently seen a couple of references to 3.1.2 go by in
>> checkins which made me wonder whether dates have been proposed yet
>> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
>> didn't see any references in the PEPs.  Some advance warning would
>> be nice.
> 
> There are a couple of release blockers right now. Once they are fixed
> or deferred, I think it would be nice to have a 3.1.2. Why do you
> need "some advance warning" though?

Advance warning does allow interested users that would consider
upgrading to schedule time for testing before the maintenance release
comes out. This is particularly useful in helping to make a 1-week RC
period effective in picking up issues that might otherwise lead to a
brown paper bag release to fix major issues that slipped through our own
automated test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 19:01:37 2010
From: nad at acm.org (Ned Deily)
Date: Sun, 17 Jan 2010 10:01:37 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
	<4B53135A.7060104@gmail.com>
Message-ID: <nad-25165C.10013717012010@ger.gmane.org>

In article <4B53135A.7060104 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> I've recently seen a couple of references to 3.1.2 go by in
> >> checkins which made me wonder whether dates have been proposed yet
> >> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
> >> didn't see any references in the PEPs.  Some advance warning would
> >> be nice.
> > There are a couple of release blockers right now. Once they are fixed
> > or deferred, I think it would be nice to have a 3.1.2. Why do you
> > need "some advance warning" though?
> 
> Advance warning does allow interested users that would consider
> upgrading to schedule time for testing before the maintenance release
> comes out. This is particularly useful in helping to make a 1-week RC
> period effective in picking up issues that might otherwise lead to a
> brown paper bag release to fix major issues that slipped through our own
> automated test coverage.

That. and resource contention: there are always potential fixes in the 
pipeline that could or should be bumped in priority if one knows there 
is a code cutoff approaching.

-- 
 Ned Deily,
 nad at acm.org



From ziade.tarek at gmail.com  Sun Jan 17 20:51:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 20:51:58 +0100
Subject: [Python-Dev] Enhancing the shutil module
Message-ID: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>

Hello,

For 2.7/3.2, I am in the process of removing modules in Distutils that
can be replaced by calls to existing functions in stdlib. For
instance, "dir_util" and "file_util" (old modules from the Python 1.x
era) are going away in favor of calls to shutil (and os), so the
Distutils package gets lighter.

Another module I would like to move away from Distutils is
"archive_util". It contains helpers to build archives, whether they
are zip or tar files. I propose to move those useful functions into
shutil, as this seems the most logical place.

I also propose to maintain this "shutil" module for now on (no one is
declared as a maintainer in maintainers.rst) since Distutils will
become a heavy user of its functions.

Any objections/opinions ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 20:53:03 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 19:53:03 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <4B536A9F.5050203@voidspace.org.uk>

On 17/01/2010 19:51, Tarek Ziad? wrote:
> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>
> Regards,
> Tarek
>
>    
I think it's a great idea. :-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From brett at python.org  Sun Jan 17 20:55:47 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jan 2010 11:55:47 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>

On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>

If it's archive-agnostic then shutil is probably the best place.


>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
>
Great!


> Any objections/opinions ?
>

No objections from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100117/f97ac6eb/attachment-0003.htm>

From solipsis at pitrou.net  Sun Jan 17 21:04:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 20:04:41 +0000 (UTC)
Subject: [Python-Dev] Enhancing the shutil module
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <loom.20100117T210307-719@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.

Are these functions portable? Do they rely on external programs?

> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.

It's ok for me.

Regards

Antoine.




From ziade.tarek at gmail.com  Sun Jan 17 21:09:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:09:18 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
Message-ID: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>

On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Hello,
>>
>> For 2.7/3.2, I am in the process of removing modules in Distutils that
>> can be replaced by calls to existing functions in stdlib. For
>> instance, "dir_util" and "file_util" (old modules from the Python 1.x
>> era) are going away in favor of calls to shutil (and os), so the
>> Distutils package gets lighter.
>>
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>
> If it's archive-agnostic then shutil is probably the best place.

In more details:
It allows the creation of gzip, bzip2, tar and zip files through a single API.
There's a registry of supported formats and the API is driven by a
format identifier.

To do the work it uses stdlib's compression modules. Although it tries
the "zip" system command as a fallback if the "zipfile" module is not
present.

(notice that I've removed the support of "compress" (.Z) some time ago)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 21:15:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 20:15:00 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <loom.20100117T210307-719@post.gmane.org>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
Message-ID: <4B536FC4.9090304@voidspace.org.uk>

On 17/01/2010 20:04, Antoine Pitrou wrote:
> Tarek Ziad?<ziade.tarek<at>  gmail.com>  writes:
>    
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>>      
> Are these functions portable? Do they rely on external programs?
>
>    

I believe that part of the work that Tarek has been doing has been to 
make these distutils commands use the Python standard library and not 
depend on external programs. In which case they seem like *excellent* 
additions to the shutil module.

Of course Tarek can speak for himself...

Michael

>> I also propose to maintain this "shutil" module for now on (no one is
>> declared as a maintainer in maintainers.rst) since Distutils will
>> become a heavy user of its functions.
>>      
> It's ok for me.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Sun Jan 17 21:43:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:43:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B536FC4.9090304@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
Message-ID: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:15 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>> Are these functions portable? Do they rely on external programs?
>>
>>
>
> I believe that part of the work that Tarek has been doing has been to make
> these distutils commands use the Python standard library and not depend on
> external programs. In which case they seem like *excellent* additions to the
> shutil module.

yes, in the past the "tar" files where created using the "tar" command
but this has been changed. For a while now, they are portable and they
use stdlib code only. A recent addition is to be able to define
user/group permissions in the tar files, thanks to Lars' work in the
tarfile module.

There's one remaining external call for "zip" done if the zip module
is not found, but I am happy to remove it and throw an exception if
it's not found, and keep the external "zip" call on Distutils side, so
shutil stays 100% stdlib-powered.

> Of course Tarek can speak for himself...

Thanks for explaining it ! :)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From sridharr at activestate.com  Sun Jan 17 22:50:52 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 17 Jan 2010 13:50:52 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
Message-ID: <4B53863C.5060304@activestate.com>

On 1/17/2010 12:09 PM, Tarek Ziad? wrote:
> On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon<brett at python.org>  wrote:
>> >  On Sun, Jan 17, 2010 at 11:51, Tarek Ziad?<ziade.tarek at gmail.com>  wrote:
>>> >>  Another module I would like to move away from Distutils is
>>> >>  "archive_util". It contains helpers to build archives, whether they
>>> >>  are zip or tar files. I propose to move those useful functions into
>>> >>  shutil, as this seems the most logical place.
>> >  If it's archive-agnostic then shutil is probably the best place.
> In more details:
> It allows the creation of gzip, bzip2, tar and zip files through a single API.
> There's a registry of supported formats and the API is driven by a
> format identifier.

Will it also allow decompression of the said archive types? Distribute 
has some utility code to handle zip/tar archives. So does PyPM. This is 
because the `tarfile` and `zipfile` modules do not "just work" due to 
several issues.

See http://gist.github.com/279606

Take note of the following in the above code:

  1) _ensure_read_write_access
  2) *File.is_valid
  3) ZippedFile.extract ... issue 6510
  4) ZippedFile.extract ... issue 6609
  5) TarredFile.extract ... issue 6584
  6) The way unpack() detects the unpacked directory.

-srid


From ziade.tarek at gmail.com  Sun Jan 17 23:09:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 23:09:29 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B53863C.5060304@activestate.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
	<4B53863C.5060304@activestate.com>
Message-ID: <94bdd2611001171409j4ec1e0bfj47e59894fdec0d8a@mail.gmail.com>

On Sun, Jan 17, 2010 at 10:50 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
[..]
> Will it also allow decompression of the said archive types?

No but it would make sense having this one as well.
Distribute/Setuptools' "unpack_archive" (used by easy_install) was
implemented using the same principle as Distutils' "make_archive".

I can add it as well while I am at it : it has been successfully used
for years by easy_install.

> Distribute has
> some utility code to handle zip/tar archives. So does PyPM. This is because
> the `tarfile` and `zipfile` modules do not "just work" due to several
> issues.
>
> See http://gist.github.com/279606
>
> Take note of the following in the above code:
>
> ?1) _ensure_read_write_access
> ?2) *File.is_valid
> ?3) ZippedFile.extract ... issue 6510
> ?4) ZippedFile.extract ... issue 6609
> ?5) TarredFile.extract ... issue 6584

Looks like some of these are already fixed (I just looked quickly in
the bug tracker though)
If its not already done and pending, it would be great if you could
refactor your fixes into patches for the remaining issues for each one
of those modules

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From barry at python.org  Sun Jan 17 23:56:37 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Jan 2010 17:56:37 -0500
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <4B52D6AD.6090302@v.loewis.de>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<4B52D6AD.6090302@v.loewis.de>
Message-ID: <BEC881FD-2BDE-4B5C-A570-B8C1E23D6DE1@python.org>

On Jan 17, 2010, at 4:21 AM, Martin v. L?wis wrote:

> Barry was once proposing time-based releases; this idea didn't really
> catch on.

I'm still a proponent of this, but it doesn't seem like there's enough support for it.

> It's the release manager who decides when the next bug fix release will
> be made, often in response to developers asking for one.

I'm happy to make a 2.6.5 release whenever it makes sense.
-Barry



From ziade.tarek at gmail.com  Fri Jan  1 16:07:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:07:20 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04D81D51@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<94bdd2610912141458m52da21ear4970506a37214e6@mail.gmail.com>
	<4dab5f760912230700l38a597f7l855bb2304025d6d5@mail.gmail.com>
	<F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
Message-ID: <94bdd2611001010707h4043ff64x7476049e435852b@mail.gmail.com>

2009/12/23 Glyph Lefkowitz <glyph at twistedmatrix.com>:
[..]
> 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages".
>
> The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. ?In other words it's got "bin", "lib", "share", "etc", etc.. ?~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation.

Right, good idea

Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Fri Jan  1 16:32:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:32:27 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
Message-ID: <94bdd2611001010732o38a563bcu6d0cc9122ed5c88f@mail.gmail.com>

On Sat, Dec 12, 2009 at 9:02 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> A while ago I've proposed to refactor the APIs that provides access to
> the installation paths and configuration variables at runtime into a
> single module called "sysconfig", and make it easier for all
> implementations to work with them.

I've applied the suggestions made in this thread.

If no one objects, here's what I am going to do now:

I'll merge this change in the trunk, (I'll drop the branch changesets
in favor of one single, clean changeset) and start to work on the
corresponding doc, so more people will be able to give some feedback
on the APIs once the second 2.7 alpha is out.

Then, in a second step, I'll work on the removal of the Makefile and
pyconfig.h dependencies by adding a _synconfig.py file as suggested
earlier, that will be created during the build process.

Regards,
Tarek


From status at bugs.python.org  Fri Jan  1 18:07:11 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jan 2010 18:07:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100101170711.A5AAF78654@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/25/09 - 01/01/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2537 open (+22) / 16902 closed (+19) / 19439 total (+41)

Open issues with patches:  1016

Average duration of open issues: 705 days.
Median duration of open issues: 462 days.

Open Issues Breakdown
   open  2504 (+22)
pending    32 ( +0)

Issues Created Or Reopened (42)
_______________________________

doc: Code is not always colored                                  12/30/09
CLOSED http://bugs.python.org/issue7487    reopened flox                          
                                                                               

documention buglet for PyBuffer                                  12/26/09
CLOSED http://bugs.python.org/issue7577    created  ronaldoussoren                
       easy                                                                    

Behavior of operations on a closed file object is not documented 12/26/09
       http://bugs.python.org/issue7578    created  blep                          
                                                                               

Patch to add docstrings to msvcrt                                12/26/09
       http://bugs.python.org/issue7579    created  brian.curtin                  
       patch                                                                   

distutils makefile from python.org installer doesn't work on OS  12/27/09
       http://bugs.python.org/issue7580    created  bskaplan                      
                                                                               

incomplete doc of zlib                                           12/27/09
CLOSED http://bugs.python.org/issue7581    created  coolwanglu                    
                                                                               

[patch] diff.py to use iso timestamp                             12/27/09
       http://bugs.python.org/issue7582    created  techtonik                     
       patch                                                                   

doctest should normalize tabs when comparing output              12/27/09
       http://bugs.python.org/issue7583    created  techtonik                     
                                                                               

datetime.rfcformat() for Date and Time on the Internet           12/27/09
       http://bugs.python.org/issue7584    created  techtonik                     
                                                                               

[patch] difflib should separate filename from timestamp with tab 12/27/09
       http://bugs.python.org/issue7585    created  techtonik                     
       patch                                                                   

Typo in collections documentation                                12/28/09
CLOSED http://bugs.python.org/issue7586    created  nneonneo                      
                                                                               

Python 3.1.1 source build issues                                 12/28/09
CLOSED http://bugs.python.org/issue7587    created  sharmaag                      
                                                                               

unittest.TestCase.shortDescription isn't short anymore           12/28/09
       http://bugs.python.org/issue7588    created  exarkun                       
                                                                               

setup.py shouldn't try to build nis module when nis headers aren 12/28/09
CLOSED http://bugs.python.org/issue7589    created  Arfrever                      
       patch                                                                   

'exceptions' module mentioned in documentation                   12/28/09
CLOSED http://bugs.python.org/issue7590    created  July                          
                                                                               

test_get_platform fails on 3.1                                   12/28/09
       http://bugs.python.org/issue7591    created  pitrou                        
       buildbot                                                                

ssl module documentation: SSLSocket.unwrap description shown twi 12/28/09
       http://bugs.python.org/issue7592    created  mnewman                       
                                                                               

Computed-goto patch for RE engine                                12/29/09
       http://bugs.python.org/issue7593    created  akuchling                     
       patch, needs review                                                     

shlex refactoring                                                12/29/09
       http://bugs.python.org/issue7594    created  ferringb                      
       patch, needs review                                                     

Doc typo for select.kevent()                                     12/29/09
CLOSED http://bugs.python.org/issue7595    created  whit537                       
                                                                               

test_logging sometimes fails                                     12/29/09
CLOSED http://bugs.python.org/issue7596    created  pitrou                        
                                                                               

curses.use_env implementation error                              12/30/09
       http://bugs.python.org/issue7597    created  kanru                         
       patch                                                                   

Cannot install, problems with assembly?                          12/30/09
CLOSED http://bugs.python.org/issue7598    created  sh.00                         
                                                                               

(c)ElementTree can produce invalid XML                           12/30/09
       http://bugs.python.org/issue7599    created  jwilk                         
                                                                               

interpreter crashes before start                                 12/30/09
CLOSED http://bugs.python.org/issue7600    created  mhh91                         
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc         12/30/09
CLOSED http://bugs.python.org/issue7601    created  sadhak                        
                                                                               

Doc: make clean and make update do not delete or update Doc/tool 12/30/09
CLOSED http://bugs.python.org/issue7602    created  flox                          
                                                                               

There is no 'seq' type                                           12/30/09
CLOSED http://bugs.python.org/issue7603    created  tom66                         
                                                                               

delattr __slots__ inconsistancy                                  12/30/09
CLOSED http://bugs.python.org/issue7604    created  ferringb                      
                                                                               

test_cmd_line fails with non-ascii path                          12/30/09
       http://bugs.python.org/issue7605    created  pitrou                        
                                                                               

test_xmlrpc fails with non-ascii path                            12/30/09
       http://bugs.python.org/issue7606    created  pitrou                        
                                                                               

stringlib fastsearch could be improved on 64-bit builds          12/30/09
       http://bugs.python.org/issue7607    created  pitrou                        
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.             12/30/09
CLOSED http://bugs.python.org/issue7608    created  alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                   12/31/09
CLOSED http://bugs.python.org/issue7609    created  Arfrever                      
       patch                                                                   

Cannot use both read and readline method in same ZipExtFile obje 12/31/09
       http://bugs.python.org/issue7610    created  lucifer                       
                                                                               

shlex not posix compliant when parsing "foo#bar"                 12/31/09
       http://bugs.python.org/issue7611    created  jjdmol2                       
                                                                               

Fix "pass and object" typo in Library Reference / Built-in Types 12/31/09
CLOSED http://bugs.python.org/issue7612    created  ygale                         
       patch                                                                   

[cppcheck] found a regression : Invalid number of character ((). 12/31/09
CLOSED http://bugs.python.org/issue7613    created  ettl.martin                   
       patch                                                                   

Python 2.6.4 segfaults                                           12/31/09
CLOSED http://bugs.python.org/issue7614    created  ttsiod                        
                                                                               

unicode_escape codec does not escape quotes                      01/01/10
       http://bugs.python.org/issue7615    created  rhansen                       
       patch                                                                   

test_memoryview test_setitem_writable failures with Intel ICC    01/01/10
       http://bugs.python.org/issue7616    created  ivank                         
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option 01/01/10
       http://bugs.python.org/issue7617    created  Arfrever                      
       patch                                                                   



Issues Now Closed (46)
______________________

True division of integers could be more accurate                  715 days
       http://bugs.python.org/issue1811    mark.dickinson                
       patch                                                                   

API to clear most free lists                                      690 days
       http://bugs.python.org/issue2019    amaury.forgeotdarc            
       patch                                                                   

unit test UnicodeWarning                                          687 days
       http://bugs.python.org/issue2100    ezio.melotti                  
                                                                               

test_disutils fails                                               568 days
       http://bugs.python.org/issue3054    ronaldoussoren                
       patch                                                                   

test_formatdate_usegmt fails on non-englisch locale               451 days
       http://bugs.python.org/issue4046    r.david.murray                
                                                                               

"??" in u"foo" raises a misleading error                          410 days
       http://bugs.python.org/issue4328    r.david.murray                
                                                                               

zipfile - add __exit__ attribute to make ZipFile object compatib  287 days
       http://bugs.python.org/issue5511    ezio.melotti                  
       patch                                                                   

test_urllib2 fails - urlopen error file not on local host         271 days
       http://bugs.python.org/issue5625    orsenthil                     
                                                                               

Improve --with-dbmliborder option                                 170 days
       http://bugs.python.org/issue6491    benjamin.peterson             
       patch                                                                   

Move the special-case for integer objects out of PyBytes_FromObj  141 days
       http://bugs.python.org/issue6687    alexandre.vassalotti          
       patch, 26backport                                                       

setup.py fails to find headers of system libffi                   104 days
       http://bugs.python.org/issue6943    benjamin.peterson             
       patch, easy                                                             

The replacement suggested for callable(x) in py3k is not	equival   92 days
       http://bugs.python.org/issue7006    benjamin.peterson             
       patch                                                                   

C/API - Document exceptions                                        89 days
       http://bugs.python.org/issue7033    lekma                         
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi   35 days
       http://bugs.python.org/issue7381    georg.brandl                  
       patch                                                                   

optparse Documentation References Example Files that do not Exis   30 days
       http://bugs.python.org/issue7404    georg.brandl                  
       patch                                                                   

datetime.datetime.isoformat truncation problem                     29 days
       http://bugs.python.org/issue7413    amaury.forgeotdarc            
       patch, needs review                                                     

doc: Code is not always colored                                     0 days
       http://bugs.python.org/issue7487    georg.brandl                  
                                                                               

float('nan')**2 != nan. float('nan')**2 error 33 on windows        13 days
       http://bugs.python.org/issue7534    mark.dickinson                
       patch                                                                   

If a generator raises TypeError when being unpacked, an unrelate   10 days
       http://bugs.python.org/issue7548    amaury.forgeotdarc            
                                                                               

ctypes doc improvement: c_char_p                                    6 days
       http://bugs.python.org/issue7569    georg.brandl                  
                                                                               

Strange issue : cursor.commit() with sqlite                         5 days
       http://bugs.python.org/issue7572    ghaering                      
                                                                               

tes_math fails Mac OS X 10.4 due to OverflowError in test_mtestf    5 days
       http://bugs.python.org/issue7575    mark.dickinson                
       patch                                                                   

documention buglet for PyBuffer                                     2 days
       http://bugs.python.org/issue7577    georg.brandl                  
       easy                                                                    

incomplete doc of zlib                                              0 days
       http://bugs.python.org/issue7581    amaury.forgeotdarc            
                                                                               

Typo in collections documentation                                   0 days
       http://bugs.python.org/issue7586    georg.brandl                  
                                                                               

Python 3.1.1 source build issues                                    0 days
       http://bugs.python.org/issue7587    amaury.forgeotdarc            
                                                                               

setup.py shouldn't try to build nis module when nis headers aren    1 days
       http://bugs.python.org/issue7589    benjamin.peterson             
       patch                                                                   

'exceptions' module mentioned in documentation                      1 days
       http://bugs.python.org/issue7590    georg.brandl                  
                                                                               

Doc typo for select.kevent()                                        0 days
       http://bugs.python.org/issue7595    georg.brandl                  
                                                                               

test_logging sometimes fails                                        1 days
       http://bugs.python.org/issue7596    ezio.melotti                  
                                                                               

Cannot install, problems with assembly?                             0 days
       http://bugs.python.org/issue7598    ezio.melotti                  
                                                                               

interpreter crashes before start                                    0 days
       http://bugs.python.org/issue7600    r.david.murray                
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            1 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

Doc: make clean and make update do not delete or update Doc/tool    0 days
       http://bugs.python.org/issue7602    georg.brandl                  
                                                                               

There is no 'seq' type                                              0 days
       http://bugs.python.org/issue7603    benjamin.peterson             
                                                                               

delattr __slots__ inconsistancy                                     0 days
       http://bugs.python.org/issue7604    benjamin.peterson             
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.                0 days
       http://bugs.python.org/issue7608    alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                      1 days
       http://bugs.python.org/issue7609    benjamin.peterson             
       patch                                                                   

Fix "pass and object" typo in Library Reference / Built-in Types    0 days
       http://bugs.python.org/issue7612    ezio.melotti                  
       patch                                                                   

[cppcheck] found a regression : Invalid number of character (().    0 days
       http://bugs.python.org/issue7613    ezio.melotti                  
       patch                                                                   

Python 2.6.4 segfaults                                              0 days
       http://bugs.python.org/issue7614    mark.dickinson                
                                                                               

Fwd: Debian Bug#42318: urllib.py has problems with malformed pro 3436 days
       http://bugs.python.org/issue210849  shinnosuke                    
                                                                               

urllib / urllib2 should cache 301 redirections                   2425 days
       http://bugs.python.org/issue735515  pitrou                        
                                                                               

fast modular exponentiation                                      2084 days
       http://bugs.python.org/issue936813  mark.dickinson                
       patch                                                                   

bdist_deb - Debian packager                                       316 days
       http://bugs.python.org/issue1054967 astraw                        
       patch                                                                   

Carbon.Scrap.PutScrapFlavor                                       987 days
       http://bugs.python.org/issue1700507 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 11 Add Option to Bind to a Local IP Address in httplib.py           462 days
open    http://bugs.python.org/issue3972   

  8 fast modular exponentiation                                     2084 days
closed  http://bugs.python.org/issue936813 

  6 [patch] difflib should separate filename from timestamp with ta    5 days
open    http://bugs.python.org/issue7585   

  6 [patch] diff.py to use iso timestamp                               5 days
open    http://bugs.python.org/issue7582   

  6 Implement fastsearch algorithm for rfind/rindex                   24 days
open    http://bugs.python.org/issue7462   

  5 test_xmlrpc fails with non-ascii path                              2 days
open    http://bugs.python.org/issue7606   

  5 test_logging sometimes fails                                       1 days
closed  http://bugs.python.org/issue7596   

  5 Patch to add docstrings to msvcrt                                  6 days
open    http://bugs.python.org/issue7579   

  5 Distutils-based installer does not detect 64bit versions of Pyt  126 days
open    http://bugs.python.org/issue6792   

  5 _sha256 et al. encode to UTF-8 by default                         17 days
open    http://bugs.python.org/issue3745   




From stefan_ml at behnel.de  Sun Jan  3 09:11:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Jan 2010 09:11:16 +0100
Subject: [Python-Dev] Providing support files to assist 3.x extension
	authors
In-Reply-To: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
References: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
Message-ID: <hhpjf3$lk1$1@ger.gmane.org>

Case Vanhorsen, 20.12.2009 01:38:
> When I ported gmpy (Python to GMP multiple precision library) to
> Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I
> found the code to slightly faster and cleaner than using PyLong_AsLong
> and checking for overflow.

You might want to look at the code Cython generates for integer type 
conversions. We use specialised coercion code that gets generated 
on-the-fly to convert Python long/int from and to all sorts of C integer 
types with compile time (portability) and runtime size/value checks. 
Depending on your needs, this may or may not be faster than the above C-API 
function.

Stefan



From david.lyon at preisshare.net  Mon Jan  4 00:42:15 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 10:42:15 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
Message-ID: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>


> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>
> This new operator removes the ambiguity the original proposal had,
> without making it more
> complex for common use cases. So if you dislike it, you will need to
> propose something
> else that also fixes the ambiguity we had.

Ok.

> Environment markers
>..
>Here are some example of fields using such markers:
>
>Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'

  Requires-Dist: [Windows] pywin32 1.0+

That's simpler, shorter, and less ambiguous. Easier to
parse for package managers.

David





From python at mrabarnett.plus.com  Mon Jan  4 01:14:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 04 Jan 2010 00:14:43 +0000
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4132F3.7020905@mrabarnett.plus.com>

David Lyon wrote:
>> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>>
>> This new operator removes the ambiguity the original proposal had,
>> without making it more
>> complex for common use cases. So if you dislike it, you will need to
>> propose something
>> else that also fixes the ambiguity we had.
> 
> Ok.
> 
>> Environment markers
>> ..
>> Here are some example of fields using such markers:
>>
>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.
> 
'win32' is more specific than 'Windows' and, to me, '1.0+' means
'>=1.0', not '>1.0'.


From martin at v.loewis.de  Mon Jan  4 01:21:53 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 04 Jan 2010 01:21:53 +0100
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4134A1.5090203@v.loewis.de>

>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.

Don't you want the PEP to complete? Why this bike-shedding?

I can agree it's shorter. I can't agree that it's simpler,
or less ambiguous.

Regards,
Martin


From ssteinerx at gmail.com  Mon Jan  4 01:29:14 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 3 Jan 2010 19:29:14 -0500
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
	Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <DF433474-9F8E-40BA-B0B1-D3021CAC6317@gmail.com>


On Jan 3, 2010, at 7:21 PM, Martin v. L?wis wrote:

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>> 
>>  Requires-Dist: [Windows] pywin32 1.0+
>> 
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
> 
> Don't you want the PEP to complete? Why this bike-shedding?

Really....

Enough, already!

S



From david.lyon at preisshare.net  Mon Jan  4 01:47:56 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 11:47:56 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <1349.218.214.45.58.1262566076.squirrel@syd-srv02.ezyreg.com>


Hi Martin, Happy New Year,

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>>
>>   Requires-Dist: [Windows] pywin32 1.0+
>>
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
>
> Don't you want the PEP to complete? Why this bike-shedding?

Well, I'm just helping out by pointing out some simpler ways
as Tarek asked me. I was only answering his question. I was out
celebrating so it took longer to reply than normal.

Bike-Shedding ? Me ? which bikeshed? wanting simple?

Anyway, I'm just reading the PEPs and commenting. If there
are some alterations that can be done, lets discuss them.

> I can agree it's shorter.
> ..

Cool.

What I'd really like is a 'Code-Repository:' keyword
so that we can install programs/libraries directly into
a system.

I feel that this would really simplify the packaging
landscape, making it much easier for the scientific
community and others to do python software installs.

This would allow us to perphaps have something even
*more modern* than CPAN.

So yes, getting PEP-345 right is important to me.

Have a nice day.

David





From abpillai at gmail.com  Mon Jan  4 10:08:05 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Mon, 4 Jan 2010 14:38:05 +0530
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <8548c5f31001040108v635a78ech33521371c6c50e82@mail.gmail.com>

On Sun, Dec 27, 2009 at 11:21 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Tue, Dec 15, 2009 at 12:37 AM, Steven Bethard
> <steven.bethard at gmail.com> wrote:
> >
> > If you're only concerned about 2.X, then yes, optparse will *never* be
> > removed from 2.X. There will be a deprecation note in the 2.X
> > documentation but deprecation warnings will only be issued when the -3
> > flag is specified. Please see the "Deprecation of optparse" section of
> > the PEP:
> >
> > http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
>
 Ref http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse .

 Considering that optparse will be deprecated like 5 years from now.
 I think this point is moot. The deprecation strategy IMHO is
 perhaps way too conservative. Maybe it should be deprecated
 faster ;)




-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100104/e0cabd8c/attachment-0002.html>

From fetchinson at googlemail.com  Mon Jan  4 10:32:10 2010
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 4 Jan 2010 10:32:10 +0100
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <fbe2e2101001040132o22302d2ct72b705ec046bcc46@mail.gmail.com>

>> If you're only concerned about 2.X, then yes, optparse will *never* be
>> removed from 2.X. There will be a deprecation note in the 2.X
>> documentation but deprecation warnings will only be issued when the -3
>> flag is specified. Please see the "Deprecation of optparse" section of
>> the PEP:
>>
>> http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
> Compare:
> http://docs.python.org/library/optparse.html
> http://argparse.googlecode.com/svn/tags/r101/doc/index.html
>
> argparse should be recommended as advanced and more featured variant
> of optparse - that goes without saying, but forcing people to switch
> and annoying them with deprecation messages brings only headache.

As has been noted already nobody is forcing people to switch. Optparse
will be available as a separate package and everybody will be free to
install it and will not have any deprecation messages anywhere.

> Just
> like optparse is better getopt, the latter could also be useful for
> people coming from other languages and porting their libraries to
> Python.
>
> I would better concentrate on real code examples how argparse solves
> problems and would really want to see
> http://www.python.org/dev/peps/pep-0389/#out-of-scope-various-feature-requests
> implemented until argparse enters phase where backward incompatible
> changes in API are impossible.
>
> I would prefer to see PEP 389 as a document describing proposed
> solutions to argument parsing problems rather than petition to replace
> one library with another. So, it should display common argument
> parsing scenarios (use cases) with examples that are also useful
> recipes for documentation. I guess more than 90% people here doesn't
> have time to read argparse methods descriptions to see what they could
> be useful for.



-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From olemis at gmail.com  Mon Jan  4 15:24:05 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 4 Jan 2010 09:24:05 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
Message-ID: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>

On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
> Thanks for the quick response.
>
> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> When I made that change, I didn't know that the __unittest "hack" was
>> being used elsewhere outside of unittest, so I felt fine replacing it
>> with another. While I still consider it an implementation detail, I
>> would be ok with exposing an "official" API for this. Perhaps
>> __unittest_ignore_traceback?
>
> Well, bazaar has had the trick for a couple of years, and googling
> around now turns up some other projects using it or thinking about it:
> <http://github.com/gfxmonk/mocktest/commit/b5e94f7ee06ab627cea2c9cf90d0aeb63ee5a698>
> <http://bitbucket.org/uche/amara/changeset/eeaf69f48271/>
> <http://twistedmatrix.com/trac/ticket/4127>
>

Add `dutest` and probably `nose` [2]_ and ...

> Reinstating the old implementation (with the same name) would mean
> that existing code would keep working with Python 2.7

IOW ... if it is removed all the libraries will have to change and
check for the Py version and ... (probably not a big deal depending on
the details of the ?solution?)

> but maybe a
> discussion could start about a new, less hacky, way of doing the same
>

I am strongly -1 for modifying the classes in ?traditional? unittest
module [2]_ (except that I am strongly +1 for the package structure
WITHOUT TOUCHING anything else ...) , and the more I think about it I
am more convinced ... but anyway, this not a big deal (and in the end
what I think is not that relevant either ... :o). So ...

pass

> May not be worthwhile making life more complicated though,
> there aren't *that* many unittest-extending projects.
>

But every library extending `unittest` will do it or otherwise
not-so-useful error messages will be displayed
;o)

PS: Happy New Year ! BTW

.. [1] [feature] nosexml should omit trace frames where `__unittest...
        (http://code.google.com/p/python-nosexml/issues/detail?id=4#c0)

.. [2] Assessment of unittest 2.7 API : new features and opinions
        (http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free new ripit 1.3.3 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/KFwyUTKH0vI/


From juanfhj at gmail.com  Tue Jan  5 17:10:16 2010
From: juanfhj at gmail.com (Juan Fernando Herrera J.)
Date: Tue, 5 Jan 2010 11:10:16 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards compatibility
Message-ID: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>

How about a new python 3 release with (possibly partial) backwards
compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
software hasn't been ported to it. I'm eager to use 3, but paradoxically,
the 3 release makes me rather stuck with 2.6. Excuse me if this has been
suggested in the past.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/7839cf7b/attachment-0004.htm>

From fuzzyman at voidspace.org.uk  Tue Jan  5 17:50:15 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 05 Jan 2010 16:50:15 +0000
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <4B436DC7.8040605@voidspace.org.uk>

On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
> How about a new python 3 release with (possibly partial) backwards 
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way 
> major software hasn't been ported to it. I'm eager to use 3, but 
> paradoxically, the 3 release makes me rather stuck with 2.6. Excuse me 
> if this has been suggested in the past.
>    

I don't know about other developers, but I certainly expected Python 3 
adoption to take a few years due to libraries only gradually making the 
jump. I also expected there to be nervousness and pain around the 
migration as well.

I think in fact that libraries *are* migrating and there are lots of 
encouraging signs. As a community we need to do all we can to promote 
Python 3 adoption, but I don't think there is much reason to be worried 
(or complacent for that matter).

All the best,

Michael Foord

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/d7b6baac/attachment-0004.htm>

From brian.curtin at gmail.com  Tue Jan  5 18:21:31 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jan 2010 11:21:31 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>

On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:

> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
The proper route to take, in my opinion, is to see what 2.x libraries you
are using that are not 3.x compatible, run 2to3 on them, then run their test
suite, and see where you get. Submit a patch or two to the library and see
what happens -- it at least gets the wheels in motion.

I'm sure everyone out there would like to dive into supporting 3.x, but it's
tough to prioritize when you are up to your eyeballs with 2.x bugs in your
tracker. Look at Twisted (
http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3)
-- over 1000 issues, ~5 developers -- 3.x support won't be here tomorrow,
but it's on the horizon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/67a4e6e2/attachment-0004.htm>

From guido at python.org  Tue Jan  5 18:23:08 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jan 2010 09:23:08 -0800
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B436DC7.8040605@voidspace.org.uk>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<4B436DC7.8040605@voidspace.org.uk>
Message-ID: <ca471dc21001050923i2ca37f6ej543faf0981c43b13@mail.gmail.com>

On Tue, Jan 5, 2010 at 8:50 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
>
> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
> I don't know about other developers, but I certainly expected Python 3
> adoption to take a few years due to libraries only gradually making the
> jump. I also expected there to be nervousness and pain around the migration
> as well.
>
> I think in fact that libraries *are* migrating and there are lots of
> encouraging signs. As a community we need to do all we can to promote Python
> 3 adoption, but I don't think there is much reason to be worried (or
> complacent for that matter).
>

Michael is right, but doesn't answer the OP's implied question "why can't
3.x be made backwards compatible with 2.6?"

Unfortunately the answer isn't easy. I wish it was "because we didn't have
time to do it" (since then the solution would be simply to call for more
volunteers to help out) -- but unfortunately, it comes closer to "we have no
idea how to do it."

Py3k was designed to be backwards incompatible, because that gives the most
long-term benefit of the language improvements. (Perhaps a better way of
saying this is that in the design of language improvements,
feature-by-feature backwards compatibility was intentionally not a
requirement, even though keeping most of the language mostly unchanged *was*
a requirement.)

In principle it would be possible to be more backwards compatible at the
syntactic level, but that's not where the pain of porting code lies -- the
major hurdles are typically he new way of thinking about Unicode (bytes vs.
text strings, instead of 8-bit-strings vs. Unicode strings), and the changed
semantics of dictionary keys and related APIs. Since these issues typically
exist across modules (passing strings and dicts between modules is common),
recognizing individual modules as "2.x" vs. "3.x" isn't possible.

Note, by the way, that you won't be "stuck" on 2.6 forever -- 2.7 alpha 1
was already released.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/eaba2f9b/attachment-0004.htm>

From regebro at gmail.com  Tue Jan  5 18:52:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 18:52:20 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <319e029f1001050952o71cb29afh66445550beeb99c2@mail.gmail.com>

On Tue, Jan 5, 2010 at 17:10, Juan Fernando Herrera J.
<juanfhj at gmail.com> wrote:
> I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.

Yes. Python 3 is not what you want to use today if you write
applications. If you write libraries 2010 is the year to port, IMO.
With some luck, 2011 will be the year to start porting applications
properly. We'll see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ianb at colorstudy.com  Tue Jan  5 20:00:49 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 13:00:49 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
Message-ID: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>

On Tue, Jan 5, 2010 at 11:21 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:
>
>> How about a new python 3 release with (possibly partial) backwards
>> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
>> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
>> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
>> suggested in the past.
>>
>>
> The proper route to take, in my opinion, is to see what 2.x libraries you
> are using that are not 3.x compatible, run 2to3 on them, then run their test
> suite, and see where you get. Submit a patch or two to the library and see
> what happens -- it at least gets the wheels in motion.
>

It's not even that easy -- libraries can't apply patches for Python 3
compatibility as they usually break Python 2 compatibility.  Potentially
libraries could apply patches that make a codebase 2to3 ready, but from what
I've seen that's more black magic than straight forward updating, as such
patches have to trick 2to3 producing the output that is desired.

The only workable workflow I've seen people propose for maintaining a single
codebase with compatibility across both 2 and 3 is to use such tricks, with
aliases to suppress some 2to3 updates when they are inappropriate, so that
you can run 2to3 on install and have a single canonical Python 2 source.
 Python 2.7 won't help much (even though it is trying) as the introduction
of non-ambiguous constructions like b"" aren't compatible with previous
versions of Python and so can't be used in many libraries (support at least
back to Python 2.5 is the norm for most libraries, I think).

Also, running 2to3 on installation is kind of annoying, as you get source
that isn't itself the canonical source, so to fix bugs you have to look at
the installed source and trace it back to the bug in the original source.

I suspect a reasonable workflow might be possible with hg and maybe patch
queues, but I don't feel familiar enough with those tools to map that out.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/e43ccd78/attachment-0004.htm>

From glyph at twistedmatrix.com  Tue Jan  5 21:24:45 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 5 Jan 2010 15:24:45 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.

From regebro at gmail.com  Tue Jan  5 21:57:35 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 21:57:35 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <319e029f1001051257k3827c52ahcd115b15d9a8ece7@mail.gmail.com>

On Tue, Jan 5, 2010 at 21:24, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It seems like this is a problem to be addressed, then. ?Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

Some of it is about changing the code so 2to3 doesn't have to do ugly
things. For example, always use iteritems(), so that 2to3 knows that
items() can be used without converting it to a list, etc. Then we do
have the problems with unicode vs string vs bytes, where I can't think
of a clever solution except your no-op shims, which admittedly is
fugly . For me that tends to turn into testing everything until the
tests run on all platforms, which I guess is close to "black magic".
Don't know what to do about that.

python-incompatibility is about documenting all differences, and also
how to make code that runs under both 2.6 and 3.0 without 2to3. But I
guess it should be extended into how to spell something that is clean
in both 2.6 and 3.x.

>> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.

Ian: If you have examples of 2to3 updated that are not appropriate
(except the x.items() -> list(x.items()) they would be appreciated.

> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

In my experience it turned out to be less of a problem than I thought.
That is to some extent because the modules I've ported has had good
test coverage, and I have fixed 99.9% of the bugs by making the tests
pass. Developing with Distributes 2to3 support has then been quite
smooth and in most cases the separation between the 2.x source and the
3.x source hasn't been a problem.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan  5 22:07:23 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 22:07:23 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <4B43AA0B.5030308@v.loewis.de>

> It's not even that easy -- libraries can't apply patches for Python 3
> compatibility as they usually break Python 2 compatibility.  Potentially
> libraries could apply patches that make a codebase 2to3 ready, but from
> what I've seen that's more black magic than straight forward updating,
> as such patches have to trick 2to3 producing the output that is desired.

I wouldn't qualify it in that way. It may be necessary occasionally to
trick 2to3, but that's really a bug in 2to3 which you should report, so
that trickery is then a work-around for a bug - something that you may
have to do with other API, as well.

The "black magic" is really more in the parts that 2to3 doesn't touch
at all (because they are inherently not syntactic); these are the
problem areas Guido refers to. The "black magic" then is to make the
same code work unmodified for both 2.x and 3.x.

> The only workable workflow I've seen people propose for maintaining a
> single codebase with compatibility across both 2 and 3 is to use such
> tricks, with aliases to suppress some 2to3 updates when they are
> inappropriate

I think you misunderstand. All this is necessary, but *not* to suppress
2to3 updates. More typically, it is necessary because 2to3 leaves the
code unmodified either way.

> Also, running 2to3 on installation is kind of annoying, as you get
> source that isn't itself the canonical source, so to fix bugs you have
> to look at the installed source and trace it back to the bug in the
> original source.

That's an issue indeed, but one that I would hope that can be fixed by
improved traceback printing. It would be good if such traceback printing
could make it into 2.7.

Regards,
Martin


From martin at v.loewis.de  Tue Jan  5 22:13:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Jan 2010 22:13:07 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <4B43AB63.3000607@v.loewis.de>

> No-op constructions like 'bytes("")' could help for older versions of
> Python, though.  A very, very small runtime shim could provide
> support for these, if 2to3 could be told about it somehow.

You actually don't *need* 2to3 support for that - bytes("") can work
in either version:

2.x:
def bytes(s):
  return s

3.x:
def bytes(s)
  return s.encode("ascii")

On top of that, you *could* as for bytes("") to be converted to b"" in
3.x - and it is indeed possible to tell 2to3 about that, and you don't
even need to modify 2to3's source to do so: it can be part of your
package.

>> Also, running 2to3 on installation is kind of annoying, as you get
>> source that isn't itself the canonical source, so to fix bugs you
>> have to look at the installed source and trace it back to the bug
>> in the original source.
> 
> Given the way tracebacks are built, i.e. from filenames stored in
> .pycs rather than based on where the code was actually loaded in the
> filesystem, couldn't 2to3 could do .pyc rewriting to point at the
> original source?  Sort of like our own version of the #line
> directive? :)

I think it could, but it would be fairly flaky as the pycs
can get lost, and lose that information after regeneration. Also,
it may be confusing in other scenarios.

I'd rather have it generate separate mapper files, and change the
traceback printing to consider these (as an option).

> Seriously though, I find it hard to believe that this is a big
> problem.  The 3.x source looks pretty similar to the 2.x source, and
> it's good to look at both if you're dealing with a 3.x issue.

That's my experience as well - the only challenge is that you can't
cut-n-paste the source URL into an editor.

Regards,
Martin


From ianb at colorstudy.com  Tue Jan  5 22:39:21 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 15:39:21 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B43AA0B.5030308@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com> 
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com> 
	<4B43AA0B.5030308@v.loewis.de>
Message-ID: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>

On Tue, Jan 5, 2010 at 3:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > It's not even that easy -- libraries can't apply patches for Python 3
> > compatibility as they usually break Python 2 compatibility. ?Potentially
> > libraries could apply patches that make a codebase 2to3 ready, but from
> > what I've seen that's more black magic than straight forward updating,
> > as such patches have to trick 2to3 producing the output that is desired.
>
> I wouldn't qualify it in that way. It may be necessary occasionally to
> trick 2to3, but that's really a bug in 2to3 which you should report, so
> that trickery is then a work-around for a bug - something that you may
> have to do with other API, as well.
>
> The "black magic" is really more in the parts that 2to3 doesn't touch
> at all (because they are inherently not syntactic); these are the
> problem areas Guido refers to. The "black magic" then is to make the
> same code work unmodified for both 2.x and 3.x.

Just to clarify, the black magic I'm referring to is things like:

try:
?? ?unicode_ = unicode
except NameError:
?? ?unicode_ = str

and some other aliases like this that are unambiguous and which 2to3
won't touch (if you write them correctly). ?If the porting guide noted
all these tricks (of which several have been developed, and I'm only
vaguely aware of a few) that would be helpful. ?It's not a lot of
tricks, but the tricks are not obvious and 2to3 gets the translation
wrong pretty often without them. ?For instance, when I say str in
Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
str and unicode to str. ?That *nothing* translates to bytes by default
(AFAIK) means that people must either be living in a bytes-free world
(which sure, lots of code does) or they are using tricks not included
in 2to3 itself.


Also replying to Glyph:
> > Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.
>
> Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in
the filesystem, couldn't 2to3 could do .pyc rewriting to point at the
original source? ?Sort of like our own version of the #line directive?
:)
>
> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
then I don't currently develop any code that is "installed", I only
develop code that is directly from a source code checkout, and where
the checkout is put on the path.  I guess I could have something that
automatically builds the code on every edit, and that's not
infeasible.  It's just not fun.  So long as I have to support Python 2
(which is like forever) then adding Python 3 only makes development
that much more complicated and much less fun, with no concrete
benefits.  Which is a terribly crotchety of me.  Sorry.

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker


From martin at v.loewis.de  Tue Jan  5 23:23:53 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 23:23:53 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
Message-ID: <4B43BBF9.2000302@v.loewis.de>

> Just to clarify, the black magic I'm referring to is things like:
> 
> try:
>     unicode_ = unicode
> except NameError:
>     unicode_ = str
> 
> and some other aliases like this that are unambiguous and which 2to3
> won't touch (if you write them correctly).  If the porting guide noted
> all these tricks (of which several have been developed, and I'm only
> vaguely aware of a few) that would be helpful.  It's not a lot of
> tricks, but the tricks are not obvious and 2to3 gets the translation
> wrong pretty often without them.  For instance, when I say str in
> Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
> str and unicode to str.

No, that's not what's happening. Instead, str is not translated at all
(i.e. it's *not* translated to str - it just isn't touched).

Translating unicode to str is always correct, AFAICT.

I'm not quite sure what the magic above is supposed to achieve: in 2.x,
unicode_ becomes an alias to unicode, in 3.x, 2to3 translates unicode
to str, and then the block becomes

try:
  unicode_ = str
except NameError:
  unicode_ = str

so the NameError is actually never triggered.

Could it be that the magic is proposed for a mode where you *don't*
use 2to3?

> That *nothing* translates to bytes by default
> (AFAIK) means that people must either be living in a bytes-free world
> (which sure, lots of code does) or they are using tricks not included
> in 2to3 itself.

By your above definition of "translates", the function "bytes" is
translated to bytes (i.e. it isn't touched by 2to3).

> Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
> then I don't currently develop any code that is "installed", I only
> develop code that is directly from a source code checkout, and where
> the checkout is put on the path.  I guess I could have something that
> automatically builds the code on every edit, and that's not
> infeasible.  It's just not fun.

Depends on where you draw your fun from. It is indeed fun to me, but
I can see why it may not be fun to you. So you could ask me to do it for
you - or you may try to use what I have already done for you, so you
don't have to do it.

> So long as I have to support Python 2
> (which is like forever) then adding Python 3 only makes development
> that much more complicated and much less fun, with no concrete
> benefits.

I doubt it will be *much* more complicated - only a little.
As for concrete benefits: there may be no benefits at this point,
but in the long run, starting now will have saved you a lot of
pressure in the long run, and stop users from switching away from
your packages because of lack of Python 3 support.

Regards,
Martin


From ziade.tarek at gmail.com  Wed Jan  6 01:08:34 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 01:08:34 +0100
Subject: [Python-Dev] PEP 386 and PEP 345
Message-ID: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>

Hi,

I think we've reached a consensus on those two PEPs.

Although, there's one last point that was forgotten in the discussions
: I've introduced "rc" in the pre-releases markers, so PEP 386 is
compatible with Python's own version scheme.  "rc" comes right after
"c" in the sorting. It's slightly redundant with the "c" marker but I
don't think this really matters as long as consumers know how to order
them (a < b < c < rc). I have also stated that "c" is the preferred
marker for third party projects, from PEP 386 point of view.

Is there anything else I can do to make those two PEPs accepted ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From david.lyon at preisshare.net  Wed Jan  6 01:26:34 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:26:34 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <4B43BBF9.2000302@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
	<4B43BBF9.2000302@v.loewis.de>
Message-ID: <1322.218.214.45.58.1262737594.squirrel@syd-srv02.ezyreg.com>

Hi Martin,

> ... but in the long run, starting now will have saved you a lot of
> pressure in the long run, and stop users from switching away from
> your packages because of lack of Python 3 support.

In a production situation it works the other way around. If there's
an application that requires twisted (or whatever package) then most
people would use the appropriate interpreter to match the library.

Since you guys all did your jobs so well :-) doing so is painless.

Because there is so much "comfort" with the existing situation it
makes it an effort for people to move to a different lounge chair.
Namely python 3.

I'd suggest that moving the package set (pypi) to python 3 could
be kicked along with the help of some automated tools. I don't
know what tools you guys have got.

But I am very sure that if code analysis was provided to package
developers on python 3 (so they don't have to run it themselves),
then it would be like an even bigger tv screen in a bigger lounge
room and it would assist in drawing them over.

David







From david.lyon at preisshare.net  Wed Jan  6 01:36:24 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:36:24 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <1337.218.214.45.58.1262738184.squirrel@syd-srv02.ezyreg.com>


Hi Ian,

> The only workable workflow I've seen people propose for maintaining a
> single
> codebase with compatibility across both 2 and 3 is to use such tricks,
> with
> aliases to suppress some 2to3 updates when they are inappropriate, so that
> you can run 2to3 on install and have a single canonical Python 2 source.
>  Python 2.7 won't help much (even though it is trying) as the introduction
> of non-ambiguous constructions like b"" aren't compatible with previous
> versions of Python and so can't be used in many libraries (support at
> least
> back to Python 2.5 is the norm for most libraries, I think).
>
> Also, running 2to3 on installation is kind of annoying, as you get source
> that isn't itself the canonical source, so to fix bugs you have to look at
> the installed source and trace it back to the bug in the original source.
>
> I suspect a reasonable workflow might be possible with hg and maybe patch
> queues, but I don't feel familiar enough with those tools to map that out.

That's why I am saying that we need a Code-Repository: section in PEP-345
Metadata.

Let package developers keep two branches in SCM. One for 2.x and one for 3.x

Let them backport features from their 3.x development series to their 2.x
code base. In exactly the same way that it is done in so many other
developments today.

Keeping multiple branches of code is a very common thing these days. And
there are tools in the outside world (bzr,svn,mercurial) that allow us to
do it very easily.

Let's have that support in python; it will make life easier.

David










From david.lyon at preisshare.net  Wed Jan  6 03:01:22 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 13:01:22 +1100 (EST)
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <1617.218.214.45.58.1262743282.squirrel@syd-srv02.ezyreg.com>

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?

Tarek,

Given that I helped out so much last year with the PEP in discussing
different options, even if they weren't accepted, I really feel that it is
unfair if my name isn't mentioned. It was a huge time sacrifice on my
part.

For example, even if I only managed to explain the version numbering and
clarify how that worked. It did take me some time to do that.

What I did do however, was spend a lot of time with the multiplatform
"Markers". I still think that two short weeks more of "discussion" could
resolve some issues. That discussion went for 4 months on distutils-ml.

Look, major issues aside, can you make the following concessions on
PEP-345 which I only feel will help it, namely:

 1) Source-Repository: specify a code repository to install from

 2) Streamline Requires-Python: by implementing "Markers" as
    noted by the PEP. A marker being something like
    "Requires-Python(windows): lxml". Otherwise remove the
    word marker from the PEP and just replace with "metacode".
    Markers are what were discussed on distutils-ml. Metacode
    is what is described in the PEP.

 3) Remove the inconsistency and platform ambiguities. Especially
    for windows users. For example, "win32" is extremely confusing
    for windows users right now. As more and more systems now are
    64 bit. Use the platform module, instead of the sys module
    for constants. I'll post to distutils-ml on this.

I am certainly not trying to hold this PEP up, and I apologise on
my part for my late attention. I will post to distutils-ml on these
and i promise to keep my comments unheated and unwitty.

Having said that, PEP-345 is *super-important*. A week or two or three
more discussion and the issues can be resolved.

We all just want to focus on being productive. It would be a great
accomplishment for you to get PEP-345 approved and likewise for
me getting mentioned even in a minor role as helping out on a PEP.

So I'm hoping that you can make a few last minute concessions
meaning that we can all happily go on our way in 2010.

Regards

David













From brett at python.org  Wed Jan  6 06:20:30 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Jan 2010 21:20:30 -0800
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <bbaeab101001052120qe888df8o7eb03a61e6c030c7@mail.gmail.com>

On Tue, Jan 5, 2010 at 16:08, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?
>

As you said, consensus has been reached, so just Guido's BDFL stamp of
approval is all I can think of.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/a76cb75b/attachment-0004.htm>

From chris at simplistix.co.uk  Wed Jan  6 12:19:45 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:19:45 +0000
Subject: [Python-Dev] bug triage
Message-ID: <4B4471D1.9020707@simplistix.co.uk>

Hi All,

Is there a high volume of incoming bugs to the Python tracker?
If so, I'd like to help with triaging. I think I have all the necessary 
access, what I'm missing is the knowledge of how to set myself up to get 
notifications of new bugs...

How do I do that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From fuzzyman at voidspace.org.uk  Wed Jan  6 12:24:37 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 06 Jan 2010 11:24:37 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <4B4472F5.8000709@voidspace.org.uk>

On 06/01/2010 11:19, Chris Withers wrote:
> Hi All,
>
> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the 
> necessary access, what I'm missing is the knowledge of how to set 
> myself up to get notifications of new bugs...
>
> How do I do that?
>

Bug triaging is one of Python's "big needs" and anything you do to help 
on this score would be much appreciated. Particularly reviewing new and 
outstanding issues.

I assumed there would be RSS feeds for bug tracker activity but can't 
easily find these on the tracker. There is a bot that posts activity to 
#python-dev, so there must be some way of getting this information.

All the best,

Michael



> cheers,
>
> Chris
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From solipsis at pitrou.net  Wed Jan  6 12:25:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 11:25:16 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <loom.20100106T122258-896@post.gmane.org>


Hi Chris,

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary 
> access, what I'm missing is the knowledge of how to set myself up to get 
> notifications of new bugs...

Do you really want to get such notifications? There may be a lot of them.
If you want however, you can join #python-dev on IRC (irc.freenode.net) where
there's a bot which posts updates of all bugs on the tracker. There's usually
not a lot of discussion going on so you probably won't feel flooded.

In addition to bug triage, what is needed is reviewing of existing patches, as
well as writing patches for issues which haven't been addressed yet :-)

Regards

Antoine.




From chris at simplistix.co.uk  Wed Jan  6 12:30:40 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:30:40 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B447460.7080100@simplistix.co.uk>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't 
> easily find these on the tracker. There is a bot that posts activity to 
> #python-dev, so there must be some way of getting this information.

Yeah, email-out is what I'm really after... I have it for my own Roundup 
instance so it can't be that hard to do ;-)

Roch?, you guys host the bug tracker, right? Is there email-out set up 
for it?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ncoghlan at gmail.com  Wed Jan  6 12:37:23 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 21:37:23 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B4475F3.5040406@gmail.com>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't
> easily find these on the tracker. There is a bot that posts activity to
> #python-dev, so there must be some way of getting this information.

I'm pretty sure the bugs list is still the primary spooled notification
mechanism:
http://mail.python.org/mailman/listinfo/python-bugs-list

There are also the weekly tracker activity summaries that are posted
here to python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From chris at simplistix.co.uk  Wed Jan  6 12:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:41:28 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <4B4476E8.8050709@simplistix.co.uk>

Nick Coghlan wrote:
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

That's what I was after, thanks!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From facundobatista at gmail.com  Wed Jan  6 13:03:08 2010
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 6 Jan 2010 09:03:08 -0300
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <e04bdf311001060403y3b65c647jf82fdc26266a0efe@mail.gmail.com>

On Wed, Jan 6, 2010 at 8:19 AM, Chris Withers <chris at simplistix.co.uk> wrote:

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary
> access, what I'm missing is the knowledge of how to set myself up to get
> notifications of new bugs...

Not notifications, but maybe a way to have a higher look of them for
easy selection:

  http://www.taniquetil.com.ar/cgi-bin/pytickets.py

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ziade.tarek at gmail.com  Wed Jan  6 13:29:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:29:58 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>

On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 06/01/2010 11:19, Chris Withers wrote:
>>
>> Hi All,
>>
>> Is there a high volume of incoming bugs to the Python tracker?
>> If so, I'd like to help with triaging. I think I have all the necessary
>> access, what I'm missing is the knowledge of how to set myself up to get
>> notifications of new bugs...
>>
>> How do I do that?
>>
>
> Bug triaging is one of Python's "big needs" and anything you do to help on
> this score would be much appreciated. Particularly reviewing new and
> outstanding issues.

Another useful triage I think, is to review the oldest bugs (some of
them are > 5 years)
and remove the ones that are not relevant anymore, or duplicate with
newer entries.

Tarek

-- 
Tarek Ziad? | http://ziade.org


From chris at simplistix.co.uk  Wed Jan  6 13:31:08 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 12:31:08 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B44828C.4070201@simplistix.co.uk>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this 
with someone as a paired task for those 2 days...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Wed Jan  6 13:37:55 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:37:55 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B44828C.4070201@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B44828C.4070201@simplistix.co.uk>
Message-ID: <94bdd2611001060437s2d0242aembc669c3263773fea@mail.gmail.com>

On Wed, Jan 6, 2010 at 1:31 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Another useful triage I think, is to review the oldest bugs (some of
>> them are > 5 years)
>> and remove the ones that are not relevant anymore, or duplicate with
>> newer entries.
>
> I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this with
> someone as a paired task for those 2 days...

I'll be doing Distutils stuff but I can probably help around a bit in that task:
Distutils have quite a few old issues so I can tackle those


From roche at upfrontsystems.co.za  Wed Jan  6 13:32:39 2010
From: roche at upfrontsystems.co.za (=?ISO-8859-1?Q?Roch=E9?= Compaan)
Date: Wed, 06 Jan 2010 14:32:39 +0200
Subject: [Python-Dev] bug triage
In-Reply-To: <4B447460.7080100@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B447460.7080100@simplistix.co.uk>
Message-ID: <1262781159.2836.218.camel@didi>

On Wed, 2010-01-06 at 11:30 +0000, Chris Withers wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't 
> > easily find these on the tracker. There is a bot that posts activity to 
> > #python-dev, so there must be some way of getting this information.
> 
> Yeah, email-out is what I'm really after... I have it for my own Roundup 
> instance so it can't be that hard to do ;-)
> 
> Roch?, you guys host the bug tracker, right? Is there email-out set up 
> for it?

We do, but we don't administer it. There are a few administrators taking
care of it and you should be able to reach them by logging your request
here: http://psf.upfronthosting.co.za/roundup/meta/ or post it to the
Infrastructure mailing list: infrastructure at python.org


-- 
Roch? Compaan
Upfront Systems                   http://www.upfrontsystems.co.za



From ncoghlan at gmail.com  Wed Jan  6 13:57:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 22:57:24 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B4488B4.2070208@gmail.com>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I believe someone (Daniel Diniz, maybe?) did do a pass over those some
time in the  last 12 months, so most of the obviously irrelevant ones
that are that old should already be gone. Not to say it isn't worth
doing another pass, just saying not to get disheartened if there aren't
many that can be readily closed.

There are at least a few still kicking around just because they're
difficult to deal with (there's an ancient one to do with one of the
ways circular imports can fail that I occasionally go back and reread
before moving on to something more tractable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rdmurray at bitdance.com  Wed Jan  6 14:43:17 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 08:43:17 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4476E8.8050709@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
	<4B4476E8.8050709@simplistix.co.uk>
Message-ID: <20100106134317.3EF141FB5A6@kimball.webabinitio.net>

On Wed, 06 Jan 2010 11:41:28 +0000, Chris Withers <chris at simplistix.co.uk> wrote:
> Nick Coghlan wrote:
> > I'm pretty sure the bugs list is still the primary spooled notification
> > mechanism:
> > http://mail.python.org/mailman/listinfo/python-bugs-list
> 
> That's what I was after, thanks!

Just for completeness, there's also new-bugs-announce if you want
just *new* bug notification.  That's more for people who want to
watch for bugs they want to become nosy on, though; if you are
doing triage python-bugs-list is what you want.

Please also read http://www.python.org/dev/workflow/ if you haven't
already.  Thanks for being willing to chip in!

--David


From brian.curtin at gmail.com  Wed Jan  6 15:57:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 08:57:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4488B4.2070208@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
Message-ID: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I believe someone (Daniel Diniz, maybe?) did do a pass over those some
> time in the  last 12 months, so most of the obviously irrelevant ones
> that are that old should already be gone. Not to say it isn't worth
> doing another pass, just saying not to get disheartened if there aren't
> many that can be readily closed.
>
> There are at least a few still kicking around just because they're
> difficult to deal with (there's an ancient one to do with one of the
> ways circular imports can fail that I occasionally go back and reread
> before moving on to something more tractable).
>
> Cheers,
> Nick.
>

On the topic of bugs that can be readily closed (literally), I've recently
come across a number of issues which appear to be sitting in a patch or
review stage, but their patches have been committed and the issue remains
open. What is the best course of action there? I'd just go ahead and close
the issue myself but I don't have tracker privileges.

I'm willing to help out with another Daniel Diniz-esque triage sweep if that
would help.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/05d9ebd7/attachment-0004.htm>

From ssteinerx at gmail.com  Wed Jan  6 16:14:20 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 6 Jan 2010 10:14:20 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <7FCF8B19-3465-4392-80CC-BEAEBD926ECA@gmail.com>


On Jan 6, 2010, at 7:29 AM, Tarek Ziad? wrote:

> On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> On 06/01/2010 11:19, Chris Withers wrote:
>>> 
>>> Hi All,
>>> 
>>> Is there a high volume of incoming bugs to the Python tracker?
>>> If so, I'd like to help with triaging. I think I have all the necessary
>>> access, what I'm missing is the knowledge of how to set myself up to get
>>> notifications of new bugs...
>>> 
>>> How do I do that?
>>> 
>> 
>> Bug triaging is one of Python's "big needs" and anything you do to help on
>> this score would be much appreciated. Particularly reviewing new and
>> outstanding issues.
> 
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I was actually thinking about that the other day when I saw that the average age of bugs on the Python tracker was at some hideously large 3 digit number.  

The 'success' statistic would be to bring that down below, say, 100.

S



From skip at pobox.com  Wed Jan  6 17:38:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 6 Jan 2010 10:38:13 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <19268.48245.616046.874126@montanaro.dyndns.org>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> I'm pretty sure the bugs list is still the primary spooled
    Nick> notification mechanism:
    Nick> http://mail.python.org/mailman/listinfo/python-bugs-list

Actually, there is a new-bugs-announce list:

    http://mail.python.org/mailman/listinfo/new-bugs-announce

Skip


From solipsis at pitrou.net  Wed Jan  6 18:56:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 17:56:49 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <hi2it0$q48$1@ger.gmane.org>

Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> On the topic of bugs that can be readily closed (literally), I've
> recently come across a number of issues which appear to be sitting in a
> patch or review stage, but their patches have been committed and the
> issue remains open. What is the best course of action there?

Post a message on the issue asking for info.




From solipsis at pitrou.net  Wed Jan  6 19:09:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 18:09:39 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<hi2it0$q48$1@ger.gmane.org>
Message-ID: <loom.20100106T190650-337@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> > On the topic of bugs that can be readily closed (literally), I've
> > recently come across a number of issues which appear to be sitting in a
> > patch or review stage, but their patches have been committed and the
> > issue remains open. What is the best course of action there?
> 
> Post a message on the issue asking for info.

Ok, I realize my answer might have been a bit terse :-)
The patch might be waiting to be merged in all development branches, or it may
not totally resolve the issue, or perhaps documentation needs to be updated, or
perhaps it is pending a verdict from the buildbots, etc. You can't deduce that
the issue is completely fixed from the simple fact that something has been
committed.

Regards

Antoine Pitrou.






From brett at python.org  Wed Jan  6 20:03:32 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 11:03:32 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>  I believe someone (Daniel Diniz, maybe?) did do a pass over those some
>> time in the  last 12 months, so most of the obviously irrelevant ones
>> that are that old should already be gone. Not to say it isn't worth
>> doing another pass, just saying not to get disheartened if there aren't
>> many that can be readily closed.
>>
>> There are at least a few still kicking around just because they're
>> difficult to deal with (there's an ancient one to do with one of the
>> ways circular imports can fail that I occasionally go back and reread
>> before moving on to something more tractable).
>>
>> Cheers,
>> Nick.
>>
>
> On the topic of bugs that can be readily closed (literally), I've recently
> come across a number of issues which appear to be sitting in a patch or
> review stage, but their patches have been committed and the issue remains
> open. What is the best course of action there? I'd just go ahead and close
> the issue myself but I don't have tracker privileges.
>
>
If a core developer is willing to step forward and vouch for you to get
tracker privileges then I will give them to you. We are trying to give out
tracker privs w/ less time than required to get commit privileges. So as
long as you have helped out on a few issues in a positive and correct way
that should be enough to get one of the regulars who perform triage to
notice.

-Brett



I'm willing to help out with another Daniel Diniz-esque triage sweep if that
> would help.
>
> Brian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/f24c9595/attachment-0004.htm>

From nad at acm.org  Wed Jan  6 21:41:05 2010
From: nad at acm.org (Ned Deily)
Date: Wed, 06 Jan 2010 12:41:05 -0800
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <nad-19B167.12410506012010@news.gmane.org>

In article <4B4475F3.5040406 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't
> > easily find these on the tracker. There is a bot that posts activity to
> > #python-dev, so there must be some way of getting this information.
> 
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

Also, that mailing list (along with most python development related 
mailing lists) is mirrored at gmane.org which means it can also be 
obtained via a newsreader (NNTP) or various RSS feeds.

http://dir.gmane.org/gmane.comp.python.bugs

-- 
 Ned Deily,
 nad at acm.org



From nick.bastin at gmail.com  Wed Jan  6 22:14:54 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:14:54 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
Message-ID: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>

(This may occur on more platforms - I can test on more unix platforms
if the consensus is this is an actual problem and I'm not just a nut)

On freebsd5, if you do a simple ./configure --enable-shared in current
(2.7) trunk, your python shared library will build properly, but all
modules will fail to find the shared library and thus fail to build:

gcc -shared build/temp.freebsd-5.3-RELEASE-i386-2.7/u1/Python/Python-2.7a1/Modules/_struct.o
   -L/u1/tmp/python2.7a1/lib -L/usr/local/lib -lpython2.7 -o
build/lib.freebsd-5.3-RELEASE-i386-2.7/_struct.so
/usr/bin/ld: cannot find -lpython2.7
building '_ctypes_test' extension
...

This of course is because libpython2.7.so is in the current directory
and not (yet) installed in /usr/local/lib.  I've made a very simple
fix for this problem that works, but at least to me smells a bit
funny, which is to modify setup.py to add the following to
detect_modules():

        # If we did --enable-shared, we need to be able to find the library
        # we just built in order to build the modules.
        if platform == 'freebsd5':
            add_dir_to_list(self.compiler_obj.library_dirs, '.')


Which brings me to a few questions:

a) Does this seem like a real problem, or am I missing something obvious?

b) Does this fix seem like the sensible thing to do?  (it seems at
least that we ought to check that the user configured --enable-shared
and only set -L. in that case, if that's possible)

Setting --enable-shared when you actually have a libpython2.7.so in
/usr/local/lib (or whatever --prefix you've selected) is possibly even
more dangerous, because it may succeed in linking against a
differently-built library than what you intended.

--
Nick


From nick.bastin at gmail.com  Wed Jan  6 22:39:17 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:39:17 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <66d0a6e11001061339t38202b70mca1091e1926ecf4b@mail.gmail.com>

On Wed, Jan 6, 2010 at 16:14, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> This of course is because libpython2.7.so is in the current directory
> and not (yet) installed in /usr/local/lib.

One minor correction - as you could see from the compile line, the
actual --prefix in this case is /u1/tmp/python2.7a1, but the libraries
obviously aren't installed there yet either.  Perhaps a better fix
than setting -L. would be to put the shared library in
build/lib.freebsd-5.3-RELEASE-i386-2.7 and add that to the library
path for the linker (the build creates this directory, but installs
nothing in it).

--
Nick


From martin at v.loewis.de  Wed Jan  6 23:21:44 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Jan 2010 23:21:44 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <4B450CF8.8090009@v.loewis.de>

> b) Does this fix seem like the sensible thing to do?

No. Linking in setup.py should use the same options as if the module
was built as *shared* through Modules/Setup, which, IIUC, should use
BLDLIBRARY.

Regards,
Martin


From nick.bastin at gmail.com  Thu Jan  7 01:08:13 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 19:08:13 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B450CF8.8090009@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
Message-ID: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:21, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> b) Does this fix seem like the sensible thing to do?
>
> No. Linking in setup.py should use the same options as if the module
> was built as *shared* through Modules/Setup, which, IIUC, should use
> BLDLIBRARY.

Thanks for that pointer, that makes much more sense.  Indeed,
BLDLIBRARY on FreeBSD* is set to '-L. -lpython$(VERSION)' if you set
--enable-shared, but somehow that piece of information doesn't
propagate into the module build.  More investigation to be done...

--
Nick


From rdmurray at bitdance.com  Thu Jan  7 02:22:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 20:22:42 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
Message-ID: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>


On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On the topic of bugs that can be readily closed (literally), I've recently
> > come across a number of issues which appear to be sitting in a patch or
> > review stage, but their patches have been committed and the issue remains
> > open. What is the best course of action there? I'd just go ahead and close
> > the issue myself but I don't have tracker privileges.
> >
> >
> If a core developer is willing to step forward and vouch for you to get
> tracker privileges then I will give them to you. We are trying to give out
> tracker privs w/ less time than required to get commit privileges. So as
> long as you have helped out on a few issues in a positive and correct way
> that should be enough to get one of the regulars who perform triage to
> notice.
> 
> -Brett

I've done a quick scan of issues Brian is nosy on to refresh my
memory, and I'd say he's definitely been making positive contributions.
I'm willing to volunteer to keep an eye on his triage work for a while
if you grant him tracker privs.

Brian, I assume you'll be cognizant of Antoine's advice about making
sure a bug really should be closed before closing it :)  Hanging out in
#python-dev on freenode while working on issues can be helpful, as well,
since you can quickly ask whoever is there for second opinions on
particular bugs.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Thu Jan  7 02:28:26 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 17:28:26 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com> 
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com> 
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
Message-ID: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com> wrote:

>
> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On the topic of bugs that can be readily closed (literally), I've
> recently
> > > come across a number of issues which appear to be sitting in a patch or
> > > review stage, but their patches have been committed and the issue
> remains
> > > open. What is the best course of action there? I'd just go ahead and
> close
> > > the issue myself but I don't have tracker privileges.
> > >
> > >
> > If a core developer is willing to step forward and vouch for you to get
> > tracker privileges then I will give them to you. We are trying to give
> out
> > tracker privs w/ less time than required to get commit privileges. So as
> > long as you have helped out on a few issues in a positive and correct way
> > that should be enough to get one of the regulars who perform triage to
> > notice.
> >
> > -Brett
>
> I've done a quick scan of issues Brian is nosy on to refresh my
> memory, and I'd say he's definitely been making positive contributions.
> I'm willing to volunteer to keep an eye on his triage work for a while
> if you grant him tracker privs.
>
>
Done for the username brian.curtin (email doesn't match the one Brian
emailed from so do let me know, Brian if this is the right username).
Welcome aboard!

-Brett


> Brian, I assume you'll be cognizant of Antoine's advice about making
> sure a bug really should be closed before closing it :)  Hanging out in
> #python-dev on freenode while working on issues can be helpful, as well,
> since you can quickly ask whoever is there for second opinions on
> particular bugs.
>
> --
> R. David Murray                                      www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/726f5ca0/attachment-0004.htm>

From brian.curtin at gmail.com  Thu Jan  7 02:59:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 19:59:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
	<bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
Message-ID: <cf9f31f21001061759m4fee1cc7g2d9fe8bbfd3cb38e@mail.gmail.com>

On Wed, Jan 6, 2010 at 19:28, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com>wrote:
>
>>
>> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
>> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
>> wrote:
>> > > On the topic of bugs that can be readily closed (literally), I've
>> recently
>> > > come across a number of issues which appear to be sitting in a patch
>> or
>> > > review stage, but their patches have been committed and the issue
>> remains
>> > > open. What is the best course of action there? I'd just go ahead and
>> close
>> > > the issue myself but I don't have tracker privileges.
>> > >
>> > >
>> > If a core developer is willing to step forward and vouch for you to get
>> > tracker privileges then I will give them to you. We are trying to give
>> out
>> > tracker privs w/ less time than required to get commit privileges. So as
>> > long as you have helped out on a few issues in a positive and correct
>> way
>> > that should be enough to get one of the regulars who perform triage to
>> > notice.
>> >
>> > -Brett
>>
>> I've done a quick scan of issues Brian is nosy on to refresh my
>> memory, and I'd say he's definitely been making positive contributions.
>> I'm willing to volunteer to keep an eye on his triage work for a while
>> if you grant him tracker privs.
>>
>>
> Done for the username brian.curtin (email doesn't match the one Brian
> emailed from so do let me know, Brian if this is the right username).
> Welcome aboard!
>
>
Yep, that's the one. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/62a95fa0/attachment-0004.htm>

From python at mrabarnett.plus.com  Thu Jan  7 04:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 07 Jan 2010 03:07:56 +0000
Subject: [Python-Dev] GIL required for _all_ Python calls?
Message-ID: <4B45500C.8090905@mrabarnett.plus.com>

Hi,

I've been wondering whether it's possible to release the GIL in the
regex engine during matching.

I know that it needs to have the GIL during memory-management calls, but
does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
an easy way to find out? Or is it just a case of checking the source
files for mentions of the GIL? The header file for PyList_New, for
example, doesn't mention it!

Thanks


From john.arbash.meinel at gmail.com  Thu Jan  7 04:25:48 2010
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 06 Jan 2010 21:25:48 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B45543C.2090200@gmail.com>

MRAB wrote:
> Hi,
> 
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out? Or is it just a case of checking the source
> files for mentions of the GIL? The header file for PyList_New, for
> example, doesn't mention it!
> 
> Thanks

Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
get concurrent updating of the value, and then the final value is wrong.
(two threads do 5+1 getting 6, rather than 7, and when the decref, you
end up at 4 rather than back at 5).

AFAIK, the only things that don't require the GIL are macro functions,
like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
example, will be increfing and setting the exception state, so certainly
needs the GIL to be held.

John
=:->



From benjamin at python.org  Thu Jan  7 04:32:17 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jan 2010 21:32:17 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45543C.2090200@gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com> <4B45543C.2090200@gmail.com>
Message-ID: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>

2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
> get concurrent updating of the value, and then the final value is wrong.
> (two threads do 5+1 getting 6, rather than 7, and when the decref, you
> end up at 4 rather than back at 5).

Correct.

>
> AFAIK, the only things that don't require the GIL are macro functions,
> like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> example, will be increfing and setting the exception state, so certainly
> needs the GIL to be held.

As a general rule, I would say, no Py* macros are safe without the gil
either (the exception being Py_END_ALLOW_THREADS), since they mutate
Python objects which must be protected.



-- 
Regards,
Benjamin


From guido at python.org  Thu Jan  7 05:29:03 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Jan 2010 20:29:03 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B45543C.2090200@gmail.com> 
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
Message-ID: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>

On Wed, Jan 6, 2010 at 7:32 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> > AFAIK, the only things that don't require the GIL are macro functions,
> > like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> > example, will be increfing and setting the exception state, so certainly
> > needs the GIL to be held.
>
> As a general rule, I would say, no Py* macros are safe without the gil
> either (the exception being Py_END_ALLOW_THREADS), since they mutate
> Python objects which must be protected.

That's keeping it on the safe side, since there are some macros like
PyString_AS_STRING() that are also safe, *if* you are owning at least
one reference to the string object.

At the same time, "no Py* macros" is not quite strong enough, since if
you called PyString_AS_STRING() before releasing the GIL but you don't
own a reference to the string object, the string might be deallocated
behind your back by another thread.

A better rule would be "you may access the memory buffer in a PyString
or PyUnicode object with the GIL released as long as you own a
reference to the string object." Everything else is out of bounds (or
not worth the bother).

--
--Guido van Rossum (python.org/~guido)


From yoann.padioleau at facebook.com  Thu Jan  7 08:24:42 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Wed, 6 Jan 2010 23:24:42 -0800
Subject: [Python-Dev] relation between Python.asdl and Tools/compiler/ast.txt
Message-ID: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>

Hi,

I would like to use astgen.py to generate python classes corresponding to the 
AST of something I have defined in a .asdl file, along the line of what is
apparently done for the python AST itself. I thought astgen.py would
take as an argument a .asdl file, but apparently it instead process a file
called ast.txt. Where does this file come from ? Is it generated from
Python.asdl ?



From johan.gill at agama.tv  Thu Jan  7 10:46:52 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 10:46:52 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
Message-ID: <4B45AD8C.5000405@agama.tv>

Hi devs,
the company where I work has done some work on Python, and the question 
is how this work, owned by the company, can be contributed to the 
community properly. Are there any license issues or other pitfalls we 
need to think about? I imagine that other companies have contributed 
before, so this is probably an already solved problem.

Regards
Johan Gill
Agama Technologies



From regebro at gmail.com  Thu Jan  7 12:12:17 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 12:12:17 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45AD8C.5000405@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
Message-ID: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
> Hi devs,
> the company where I work has done some work on Python, and the question is
> how this work, owned by the company, can be contributed to the community
> properly. Are there any license issues or other pitfalls we need to think
> about? I imagine that other companies have contributed before, so this is
> probably an already solved problem.

I'm not a license lawyer, but typically your company needs to give the
code to the community. Yes, it means it stops owning it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From hodgestar+pythondev at gmail.com  Thu Jan  7 12:28:01 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 7 Jan 2010 13:28:01 +0200
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <fb73205e1001070328j44ac747fu7232a89b559ad0d9@mail.gmail.com>

On Thu, Jan 7, 2010 at 1:12 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

This is incorrect.

The correct information is at http://www.python.org/psf/contrib/.

Schiavo
Simon


From swamy.sangamesh at gmail.com  Thu Jan  7 11:46:59 2010
From: swamy.sangamesh at gmail.com (swamy sangamesh)
Date: Thu, 7 Jan 2010 16:16:59 +0530
Subject: [Python-Dev] test_ctypes failure on AIX 5.3 using python-2.6.2 and
	libffi-3.0.9
Message-ID: <c3369a531001070246s441e159cg35b4cab4e3f65541@mail.gmail.com>

Hi All,

I built the python-2.6.2 with the latest libffi-3.0.9 in AIX 5.3 using xlc
compiler.
When i try to run the ctypes test cases, two failures are seen in
test_bitfields.

*test_ints (ctypes.test.test_bitfields.C_Test) ... FAIL
test_shorts (ctypes.test.test_bitfields.C_Test) ... FAIL*

I have attached the full test case result.

If i change the type c_int and c_short to c_unit and c_ushort of class
"BITS(Structure)" in file
test_bitfields.py then no failures are seen.

Has anyone faced the similar issue or any help is appreciated.


Thanks,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0004.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctype-testcases
Type: application/octet-stream
Size: 22996 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0004.obj>

From ncoghlan at gmail.com  Thu Jan  7 13:23:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 07 Jan 2010 22:23:11 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <4B45D22F.40404@gmail.com>

Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
>> Hi devs,
>> the company where I work has done some work on Python, and the question is
>> how this work, owned by the company, can be contributed to the community
>> properly. Are there any license issues or other pitfalls we need to think
>> about? I imagine that other companies have contributed before, so this is
>> probably an already solved problem.
> 
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

As Simon pointed out, while some organisations do work that way, the PSF
isn't one of them.

The PSF only requires that the code be contributed under a license that
then allows us to turn around and redistribute it under a different open
source license without requesting additional permission from the
copyright holder. For corporate contributions, I believe the contributor
agreement needs to be signed by an authorised agent of the company - the
place to check that would probably be psf at python.org (that's the email
address for the PSF board).

Assuming the subject line relates to the code that you would like to
contribute though, that particular change is unlikely to happen - 2.6 is
in maintenance mode and changing RLock from a Python implementation to
the faster C one is solidly in new feature territory. Although a
backport of the 3.2 C RLock implementation to 2.7 could be useful, I
doubt that backporting code provided by an existing committer would be
the subject of this query :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Thu Jan  7 14:11:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:11:27 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>

On Thu, Jan 7, 2010 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder.

Even if the contributed code as in this case is a method in an
existing file? How does that work, how do they keep ownership of one
method in the threading.py method? :-)

> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)

Ah. I probably misunderstood what the suggested contribution was.
Maybe it was a separate file, which I didn't get.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Thu Jan  7 14:15:09 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 07 Jan 2010 13:15:09 +0000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
Message-ID: <4B45DE5D.3030104@voidspace.org.uk>

On 07/01/2010 13:11, Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 13:23, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>    
>> As Simon pointed out, while some organisations do work that way, the PSF
>> isn't one of them.
>>
>> The PSF only requires that the code be contributed under a license that
>> then allows us to turn around and redistribute it under a different open
>> source license without requesting additional permission from the
>> copyright holder.
>>      
> Even if the contributed code as in this case is a method in an
> existing file? How does that work, how do they keep ownership of one
> method in the threading.py method? :-)
>
>    

When contributing code to Python all work remains copyright the original 
author. The combined work is copyright *everyone*. The PSF has a license 
to distribute it, which is all that is important.

How you meaningfully exercise your ownership over chunks of code is left 
for the reader to determine...

(i.e. copyright and ownership are legal terms that don't necessarily 
mean anything *practical* in these situations.)

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From stefan_ml at behnel.de  Thu Jan  7 14:30:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 14:30:16 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
Message-ID: <hi4nl7$2ej$1@ger.gmane.org>

Guido van Rossum, 07.01.2010 05:29:
> A better rule would be "you may access the memory buffer in a PyString
> or PyUnicode object with the GIL released as long as you own a
> reference to the string object." Everything else is out of bounds (or
> not worth the bother).

Is that a "yes" regarding the OP's original question about releasing the 
GIL during regexp searches?

Stefan



From regebro at gmail.com  Thu Jan  7 14:36:42 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:36:42 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DE5D.3030104@voidspace.org.uk>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
	<4B45DE5D.3030104@voidspace.org.uk>
Message-ID: <319e029f1001070536t49f76899j111212b02c14f192@mail.gmail.com>

On Thu, Jan 7, 2010 at 14:15, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> (i.e. copyright and ownership are legal terms that don't necessarily mean
> anything *practical* in these situations.)

OK, fair enough. :-)
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Thu Jan  7 15:05:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 15:05:23 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <hi4pn2$9so$1@ger.gmane.org>

MRAB, 07.01.2010 04:07:
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER

Py_UNICODE_TOLOWER looks safe to me at first glance.


> or PyErr_SetString?

Certainly not safe.


> Is there an easy way to find out?

Release it and fix any crashes? Note that this isn't a safe solution, 
though, as some GIL requiring code may be platform specific. So a better 
approach might be to extract any obviously problematic stuff from the 
existing code (such as any exception handling, explicit ref-counting or 
object creation), and *then* try to release the GIL.

Stefan



From solipsis at pitrou.net  Thu Jan  7 16:38:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 15:38:31 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <loom.20100107T163459-842@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out?

There is no "easy way" to do so. The only safe way is to examine all the
functions or macros you want to call with the GIL released, and assess whether
it is safe to call them. As already pointed out, no reference count should be
changed, and generally no mutable container should be accessed, except if that
container is known not to be referenced anywhere else (that would be the case
for e.g. a list that your function has created and is busy populating).

I agree that releasing the GIL when doing non-trivial regex searches is a
worthwhile research, so please don't give up immediately :-)

Regards

Antoine Pitrou.




From olemis at gmail.com  Thu Jan  7 19:24:59 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 7 Jan 2010 13:24:59 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
	<24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
Message-ID: <24ea26601001071024m2abd988fuc77f94845d57483a@mail.gmail.com>

On Mon, Jan 4, 2010 at 9:24 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
>> Thanks for the quick response.
>>
>> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> but maybe a
>> discussion could start about a new, less hacky, way of doing the same
>>
>
> I am strongly -1 for modifying the classes in ?traditional? unittest
> module [2]_ (except that I am strongly +1 for the package structure
> WITHOUT TOUCHING anything else ...) , and the more I think about it I
> am more convinced ... but anyway, this not a big deal (and in the end
> what I think is not that relevant either ... :o). So ...
>

IOW, if I had all the freedom to implement it, after the pkg structure
I'd do something like :

{{{
#!python

class TestResult:
    # Everything just the same
    def _is_relevant_tb_level(self, tb):
        return '__unittest' in tb.tb_frame.f_globals

class BetterTestResult(TestResult):
    # Further code ... maybe ;o)
    #
    def _is_relevant_tb_level(self, tb):
        # This or anything else you might want to do ;o)
        #
        globs = tb.tb_frame.f_globals
        is_relevant =  '__name__' in globs and \
            globs["__name__"].startswith("unittest")
        del globs
        return is_relevant
}}}

that's what inheritance is for ;o) ... but quite probably that's not
gonna happen, just a comment .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Ubuntu sustituye GIMP por F-Spot  -
http://feedproxy.google.com/~r/simelo-es/~3/-g48D6T6Ojs/ubuntu-sustituye-gimp-por-f-spot.html


From martin at v.loewis.de  Thu Jan  7 21:24:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:24:32 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi4nl7$2ej$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
	<hi4nl7$2ej$1@ger.gmane.org>
Message-ID: <4B464300.2020204@v.loewis.de>

>> A better rule would be "you may access the memory buffer in a PyString
>> or PyUnicode object with the GIL released as long as you own a
>> reference to the string object." Everything else is out of bounds (or
>> not worth the bother).
> 
> Is that a "yes" regarding the OP's original question about releasing the
> GIL during regexp searches?

No, because the regex engine may also operate on buffers that start
moving around when you release the GIL.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:27:09 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:27:09 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B46439D.9030604@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

I don't think that's possible. The regex engine can also operate on
objects whose representation may move in memory when you don't hold
the GIL (e.g. buffers that get mutated). Even if they stay in place -
if their contents changes, regex results may be confusing.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:31:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:31:21 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
Message-ID: <4B464499.4020305@v.loewis.de>

> I would like to use astgen.py to generate python classes corresponding to the 
> AST of something I have defined in a .asdl file, along the line of what is
> apparently done for the python AST itself. I thought astgen.py would
> take as an argument a .asdl file, but apparently it instead process a file
> called ast.txt. Where does this file come from ? Is it generated from
> Python.asdl ?

astgen.py is not used to process asdl files; ast.txt lives right next to
astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

HTH,
Martin


From foom at fuhm.net  Thu Jan  7 21:36:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 7 Jan 2010 15:36:37 -0500
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B46439D.9030604@v.loewis.de>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>

On Jan 7, 2010, at 3:27 PM, Martin v. L?wis wrote:

>> I've been wondering whether it's possible to release the GIL in the
>> regex engine during matching.
>
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated). Even if they stay in place -
> if their contents changes, regex results may be confusing.

It seems probably worthwhile to optimize for the common case of using  
the regexp engine on an immutable object of type "str" or "bytes", and  
allow releasing the GIL in *that* case, even if you have to keep it  
for the general case.

James

From martin at v.loewis.de  Thu Jan  7 21:45:42 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:45:42 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
Message-ID: <4B4647F6.2060309@v.loewis.de>

>>> I've been wondering whether it's possible to release the GIL in the
>>> regex engine during matching.
>>
>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated). Even if they stay in place -
>> if their contents changes, regex results may be confusing.
> 
> It seems probably worthwhile to optimize for the common case of using
> the regexp engine on an immutable object of type "str" or "bytes", and
> allow releasing the GIL in *that* case, even if you have to keep it for
> the general case.

Right. This problem was the one that I thought of first.

Thinking about these things is fairly difficult (to me, at least), so
I think I could only tell whether I would consider a patch thread-safe
that released the GIL around matching under selected circumstances -
if I had the patch available. I don't see any obvious reason (assuming
Guido's list of conditions holds - i.e. you are holding references to
everything you access).

Regards,
Martin


From ncoghlan at gmail.com  Thu Jan  7 21:48:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 08 Jan 2010 06:48:05 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DECB.7070907@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com> <4B45DECB.7070907@agama.tv>
Message-ID: <4B464885.40806@gmail.com>

Johan Gill wrote:
> Yes, it is the new RLock implementation.
> If I understood this correctly, we should make a patch against trunk if
> anything should be contributed.

Yep.

> Do you mean that we wouldn't need the paperwork for backporting the
> original patch committed to py3k?

Whether or not a contributor agreement was essential for this particular
contribution would depend on how much new code was needed for the
backport, but the bulk of the copyright on the C RLock code would remain
with Antoine regardless.

However, sorting through the legalities of the contributor agreement
really is the best way to make sure every is squared away nice and
neatly from a legal point of view.

After all, even if I was a lawyer (which I'm not, I'm just a developer
with an interest in licensing issues), I still wouldn't be *your* lawyer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Jan  7 21:43:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 20:43:17 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <loom.20100107T214211-130@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated).

Why is it a problem? If we get a buffer through the new buffer API, the object
should ensure that the representation isn't moved away until the buffer is 
released.

Regards

Antoine.




From martin at v.loewis.de  Thu Jan  7 21:59:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:59:29 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B464B31.7040406@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

Ok, here is another problem: SRE_OP_REPEAT uses PyObject_MALLOC,
which requires the GIL (it then also may call PyErr_NoMemory,
which also requires the GIL).

Regards,
Martin


From johan.gill at agama.tv  Thu Jan  7 14:16:59 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 14:16:59 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <4B45DECB.7070907@agama.tv>

On 01/07/2010 01:23 PM, Nick Coghlan wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder. For corporate contributions, I believe the contributor
> agreement needs to be signed by an authorised agent of the company - the
> place to check that would probably be psf at python.org (that's the email
> address for the PSF board).
>
> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)
>
> Regards,
> Nick.
>
>    
Yes, it is the new RLock implementation.
If I understood this correctly, we should make a patch against trunk if 
anything should be contributed.
Do you mean that we wouldn't need the paperwork for backporting the 
original patch committed to py3k?

Regards
Johan Gill
Agama Technologies



From yoann.padioleau at facebook.com  Thu Jan  7 22:07:55 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Thu, 7 Jan 2010 13:07:55 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464499.4020305@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
Message-ID: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>


On Jan 7, 2010, at 12:31 PM, Martin v. L?wis wrote:

>> I would like to use astgen.py to generate python classes corresponding to the 
>> AST of something I have defined in a .asdl file, along the line of what is
>> apparently done for the python AST itself. I thought astgen.py would
>> take as an argument a .asdl file, but apparently it instead process a file
>> called ast.txt. Where does this file come from ? Is it generated from
>> Python.asdl ?
> 
> astgen.py is not used to process asdl files; ast.txt lives right next to
> astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

Yes, I know that. That's why I asked about the relation between ast.txt and Python.adsl.
If internally the parser uses the .adsl, but expose as a reflection mechanism things
that were generated from ast.txt, then there could be a mismatch. Where does ast.txt comes from ? Shouldn't it be generated itself from Python.adsl ?

So we would have

Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py containing all the UnarySub, Expression, classes that represents a Python AST.



> 
> HTH,
> Martin



From martin at v.loewis.de  Thu Jan  7 22:11:36 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 07 Jan 2010 22:11:36 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <loom.20100107T214211-130@post.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
Message-ID: <4B464E08.5020703@v.loewis.de>

>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated).
> 
> Why is it a problem? If we get a buffer through the new buffer API, the object
> should ensure that the representation isn't moved away until the buffer is 
> released.

In 2.7, we currently get the buffer with bf_getreadbuffer. In 3.x, we have

    /* Release the buffer immediately --- possibly dangerous
       but doing something else would require some re-factoring
    */
    PyBuffer_Release(&view);


Even if we do use the new API, and correctly, it still might be
confusing if the contents of the buffer changes underneath.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 22:16:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 22:16:12 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
Message-ID: <4B464F1C.7020404@v.loewis.de>

>> astgen.py is not used to process asdl files; ast.txt lives right
>> next to astgen.py. Instead, the asdl file is processed by
>> Parser/asdl_c.py.
> 
> Yes, I know that. That's why I asked about the relation between
> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
> expose as a reflection mechanism things that were generated from
> ast.txt, then there could be a mismatch. Where does ast.txt comes
> from ? Shouldn't it be generated itself from Python.adsl ?

What you may not be aware of is that Tools/compiler (and the
compiler package that it builds on) are both unused and unmaintained.

If the package stops working correctly - tough luck.

> So we would have
> 
> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
> containing all the UnarySub, Expression, classes that represents a
> Python AST.

No - what actually happens in Python 3.x is this: both the compiler
package and Tools/compiler are removed.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 01:10:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 01:10:35 +0100
Subject: [Python-Dev] Improve open() to support reading file starting with
	an unicode BOM
Message-ID: <201001080110.35800.victor.stinner@haypocalc.com>

Hi,

Builtin open() function is unable to open an UTF-16/32 file starting with a 
BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
file starting with a BOM, read()/readline() returns also the BOM whereas the 
BOM should be "ignored".

See recent issues related to reading an UTF-8 text file including a BOM: #7185 
(csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with 
the UTF-8-SIG encoding, but it's possible to do better.

I propose to improve open() (TextIOWrapper) by using the BOM to choose the 
right encoding. I think that only files opened in read only mode should 
support this new feature. *Read* the BOM in a *write* only file would cause 
unexpected behaviours.

Since my proposition changes the result TextIOWrapper.read()/readline() for 
files starting with a BOM, we might introduce an option to open() to enable 
the new behaviour. But is it really needed to keep the backward compatibility?

I wrote a proof of concept attached to the issue #7651. My patch only changes 
the behaviour of TextIOWrapper for reading files starting with a BOM. It 
doesn't work yet if a seek() is used before the first read.

-- 
Victor Stinner
http://www.haypocalc.com/


From guido at python.org  Fri Jan  8 01:52:20 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 16:52:20 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>

I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
talk. And for the other two, perhaps it would make more sense to have
a separate encoding-guessing function that takes a binary stream and
returns a text stream wrapping it with the proper encoding?

--Guido

On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
> file starting with a BOM, read()/readline() returns also the BOM whereas the
> BOM should be "ignored".
>
> See recent issues related to reading an UTF-8 text file including a BOM: #7185
> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
> the UTF-8-SIG encoding, but it's possible to do better.
>
> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
> right encoding. I think that only files opened in read only mode should
> support this new feature. *Read* the BOM in a *write* only file would cause
> unexpected behaviours.
>
> Since my proposition changes the result TextIOWrapper.read()/readline() for
> files starting with a BOM, we might introduce an option to open() to enable
> the new behaviour. But is it really needed to keep the backward compatibility?
>
> I wrote a proof of concept attached to the issue #7651. My patch only changes
> the behaviour of TextIOWrapper for reading files starting with a BOM. It
> doesn't work yet if a seek() is used before the first read.
>
> --
> Victor Stinner
> http://www.haypocalc.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Fri Jan  8 03:23:08 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 02:23:08 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <4B46970C.9010306@mrabarnett.plus.com>

Guido van Rossum wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
> 
Alternatively, have a universal UTF-8/16/32 encoding, ie one that 
expects UTF-8,
with or without BOM, or UTF-16/32 with BOM.
> 
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
>> See recent issues related to reading an UTF-8 text file including a BOM: #7185
>> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
>> the UTF-8-SIG encoding, but it's possible to do better.
>>
>> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
>> right encoding. I think that only files opened in read only mode should
>> support this new feature. *Read* the BOM in a *write* only file would cause
>> unexpected behaviours.
>>
>> Since my proposition changes the result TextIOWrapper.read()/readline() for
>> files starting with a BOM, we might introduce an option to open() to enable
>> the new behaviour. But is it really needed to keep the backward compatibility?
>>
>> I wrote a proof of concept attached to the issue #7651. My patch only changes
>> the behaviour of TextIOWrapper for reading files starting with a BOM. It
>> doesn't work yet if a seek() is used before the first read.
>>


From nick.bastin at gmail.com  Fri Jan  8 04:11:03 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Thu, 7 Jan 2010 22:11:03 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
Message-ID: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>

I think this problem probably needs to move over to distutils-sig, as
it doesn't seem to be specific to the way that Python itself uses
distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
linux and solaris and automatically adds '.' to the library_dirs, and
I suspect it just needs to do this on FreeBSD as well (adding bsd to
the list of platforms for which this is performed "solves" the
problem, but I don't pretend to know enough about either distutils or
freebsd to determine if this is the correct solution).

--
Nick


From glyph at twistedmatrix.com  Fri Jan  8 04:34:36 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 7 Jan 2010 22:34:36 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>



On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>> 
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".

> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

It *is* crazy, but unfortunately rather common.  Wikipedia has a good description of the issues: <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as being UTF-8, so it's become a convention to do that.  That's not good enough, so you need to guess the encoding as well to make sure, but if there is a BOM and you can otherwise verify that the file is probably UTF-8 encoded, you should discard it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/1bc40870/attachment-0004.htm>

From tseaver at palladion.com  Fri Jan  8 05:17:28 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 Jan 2010 23:17:28 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <hi6bko$d0h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
> linux and solaris and automatically adds '.' to the library_dirs, and
> I suspect it just needs to do this on FreeBSD as well (adding bsd to
> the list of platforms for which this is performed "solves" the
> problem, but I don't pretend to know enough about either distutils or
> freebsd to determine if this is the correct solution).

I wouldn't say it needed discussion on the SIG:  just create a bug
report, with the tentative patch you have worked out, and get it
assigned to Tarek.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGsdQACgkQ+gerLs4ltQ5BMQCgtV8snMXH/6dDwgdN4sIJljLd
koYAoKq6c0tKsRSrITHcygu4Od9FVzF5
=BJaE
-----END PGP SIGNATURE-----



From guido at python.org  Fri Jan  8 05:21:04 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 20:21:04 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
Message-ID: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>

On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>
>
> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
>
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>
> file starting with a BOM, read()/readline() returns also the BOM whereas the
>
> BOM should be "ignored".
>
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>
> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
> description of the issues:
> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
> being UTF-8, so it's become a convention to do that. ?That's not good
> enough, so you need to guess the encoding as well to make sure, but if there
> is a BOM and you can otherwise verify that the file is probably UTF-8
> encoded, you should discard it.

That doesn't make sense. If the file isn't UTF-8 you can't see the
BOM, because the BOM itself is UTF-8-encoded.

(And yes, I know this happens. Doesn't mean we need to auto-guess by
default; there are lots of issues e.g. what should happen after
seeking to offset 0?)

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Fri Jan  8 07:06:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 08 Jan 2010 15:06:16 +0900
Subject: [Python-Dev] Improve open() to support reading file
	starting	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
 > talk.

That doesn't stop many applications from doing it.  Python should
perhaps<wink,nudge> not produce UTF-8 + BOM without a disclaimer of
indemnification against all resulting damage, signed in blood, from
the user for each instance.

But it should do something sane when reading such files.  I can't
really see any harm in throwing it away, especially since use of
ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
IIRC.






From tseaver at palladion.com  Fri Jan  8 07:12:12 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 01:12:12 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <hi6ibr$qag$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>
>> BOM should be "ignored".
>>
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.
> 
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

The BOM should not be seekeable if the file is opened with the proposed
"guess encoding from BOM" mode:  it isn't properly part of the stream at
all in that case.

A UTF-8 BOM is an absurditiy, but it exists *everywhere* in the wild:
Python would do wll to make it as easy as possible to consume such
files, as well as the non-insane versions (UTF-16 / UTF-32 BOMs).  In
the best of all possible worlds, I would just try opening the file so:

  f = open('/path/to/file', 'r', encoding="DWIFM")

and any BOM present would set the encoding for the remainder of the stream..



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGzLsACgkQ+gerLs4ltQ5+cwCdGfycPdj6+cPfD23vH644SpHL
sI0AoLGD7nfgMEJdJhBr90yjQQHfDgcJ
=js+2
-----END PGP SIGNATURE-----



From glyph at twistedmatrix.com  Fri Jan  8 08:55:27 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 8 Jan 2010 02:55:27 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>


On Jan 7, 2010, at 11:21 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> 
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>> 
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>> 
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.  If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal.  It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I think it's pretty clear that the BOM should still be skipped in that case ...



From martin at v.loewis.de  Fri Jan  8 10:05:17 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:05:17 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <4B46F54D.9090103@v.loewis.de>

>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I think what Glyph meant is this: if a file starts with the UTF-8
signature, assume it's UTF-8. Then validate the assumption against the
rest of the file also, and then process it as UTF-8. If the rest clearly
is not UTF-8, assume that the UTF-8 signature is bogus.

I understood this proposal as a general processing guideline, not
something the io library should do (but, say, a text editor).

FWIW, I'm personally in favor of using the UTF-8 signature. If people
consider them crazy talk, that may be because UTF-8 can't possibly have
a byte order - hence I call it a signature, not the BOM. As a signature,
I don't consider it crazy at all. There is a long tradition of having
magic bytes in files (executable files, Postscript, PDF, ... - see
/etc/magic). Having a magic byte sequence for plain text to denote the
encoding is useful and helps reducing moji-bake. This is the reason it's
used on Windows: notepad would normally assume that text is in the ANSI
code page, and for compatibility, it can't stop doing that. So the UTF-8
signature gives them an exit strategy.

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:06:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:06:34 +0100
Subject: [Python-Dev] Improve open() to support reading file	starting
 with an unicode BOM
In-Reply-To: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B46F59A.5060905@v.loewis.de>

> But it should do something sane when reading such files.  I can't
> really see any harm in throwing it away, especially since use of
> ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
> IIRC.

And indeed it does, when you open the file in the utf-8-sig encoding.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 10:08:30 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 10:08:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46970C.9010306@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<4B46970C.9010306@mrabarnett.plus.com>
Message-ID: <201001081008.30904.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 03:23:08, MRAB a ?crit :
> Guido van Rossum wrote:
> > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> > talk. And for the other two, perhaps it would make more sense to have
> > a separate encoding-guessing function that takes a binary stream and
> > returns a text stream wrapping it with the proper encoding?
> 
> Alternatively, have a universal UTF-8/16/32 encoding, ie one that
> expects UTF-8,
> with or without BOM, or UTF-16/32 with BOM.

Do you mean open(filename, encoding="BOM")? I suppose that "BOM" would be a 
magical value specific to read a text file (open(filename, "r")), not a real 
codec?

Otherwise which encoding should be used for open(filename, "w", 
encoding="BOM")?

-- 
Victor Stinner
http://www.haypocalc.com/


From martin at v.loewis.de  Fri Jan  8 10:10:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:10:23 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with	an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <4B46F67F.60604@v.loewis.de>

> Builtin open() function is unable to open an UTF-16/32 file starting with a 
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
> file starting with a BOM, read()/readline() returns also the BOM whereas the 
> BOM should be "ignored".

It depends. If you use the utf-8-sig encoding, it *will* ignore the
UTF-8 signature.

> Since my proposition changes the result TextIOWrapper.read()/readline() for 
> files starting with a BOM, we might introduce an option to open() to enable 
> the new behaviour. But is it really needed to keep the backward compatibility?

Absolutely. And there is no need to produce a new option, but instead
use the existing options: define an encoding that auto-detects the
encoding from the family of BOMs. Maybe you call it encoding="sniff".

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:11:51 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:11:51 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <4B46F6D7.1050301@v.loewis.de>

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.

I'm fairly skeptical that anybody on distutils SIG is interested in
details of the Python build process...

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 11:27:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:27:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <201001081127.44044.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
(...)
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I wrote a new version of my patch (version 3):

 * don't change the default behaviour: use open(filename, encoding="BOM") to 
check the BOM is there is any
 * fix for seek(0): always ignore the BOM
 * add an unit test: check that the right encoding is detect, but also the the 
BOM is ignored (especially after a seek(0))

BOM encoding doesn't work for writing into a file, so open(filename, "w", 
encoding="BOM") raises a ValueError.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:31:37 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:31:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <201001081131.37492.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 01:52:20, Guido van Rossum a ?crit :
> And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

I choosed to modify open()+TextIOWrapper instead of writing a new function 
because I would like to avoid an extra read operation (syscall) on the file. 
With my implementation, no specific read operation is needed to detect the 
BOM. The BOM is simply checked in the first _read_chunk().

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:40:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:40:28 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Improve_open=28=29_to_support_reading?=
 =?iso-8859-1?q?_file_starting_with=09an_unicode_BOM?=
In-Reply-To: <4B46F67F.60604@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
Message-ID: <201001081140.28124.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
> > Builtin open() function is unable to open an UTF-16/32 file starting with
> > a BOM if the encoding is not specified (raise an unicode error). For an
> > UTF-8 file starting with a BOM, read()/readline() returns also the BOM
> > whereas the BOM should be "ignored".
> 
> It depends. If you use the utf-8-sig encoding, it *will* ignore the
> UTF-8 signature.

Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
remove the BOM after the first read (much harder if you use a module like 
ConfigParser or csv).

> > Since my proposition changes the result TextIOWrapper.read()/readline()
> > for files starting with a BOM, we might introduce an option to open() to
> > enable the new behaviour. But is it really needed to keep the backward
> > compatibility?
> 
> Absolutely. And there is no need to produce a new option, but instead
> use the existing options: define an encoding that auto-detects the
> encoding from the family of BOMs. Maybe you call it encoding="sniff".

Good idea, I choosed open(filename, encoding="BOM").

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Fri Jan  8 15:27:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:27:42 +0000 (UTC)
Subject: [Python-Dev] GIL required for _all_ Python calls?
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de>
Message-ID: <hi7fcu$gs7$1@ger.gmane.org>

Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
> 
> Even if we do use the new API, and correctly, it still might be
> confusing if the contents of the buffer changes underneath.

Well, no more confusing than when you compute a SHA1 hash or zlib-
compress the buffer, is it?

Regards

Antoine




From solipsis at pitrou.net  Fri Jan  8 15:34:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:34:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <loom.20100108T153305-228@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any

Well, I think if we implement this the default behaviour *should* be changed.
It looks a bit senseless to have two different "auto-choose" options, one with
encoding=None and one with encoding="BOM".

Regards

Antoine.




From guido at python.org  Fri Jan  8 16:48:44 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:48:44 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi7fcu$gs7$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de> 
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de> <hi7fcu$gs7$1@ger.gmane.org>
Message-ID: <ca471dc21001080748lbc18bbx4c4ec2d3f4f027e@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
>>
>> Even if we do use the new API, and correctly, it still might be
>> confusing if the contents of the buffer changes underneath.
>
> Well, no more confusing than when you compute a SHA1 hash or zlib-
> compress the buffer, is it?

That depends. Algorithms that make exactly one pass over the buffer
will run fine (maybe producing a meaningless result). But the regex
matcher may scan the buffer repeatedly (for backtracking purposes) and
it would take a considerable analysis to prove that cannot mess up its
internal data structures if the data underneath changes. (I give it a
decent chance that it's fine, but since it was written without ever
considering this possibility I'm not 100% sure.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:52:48 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:52:48 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
Message-ID: <ca471dc21001080752r312dd47fx32486a95336160ec@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:55 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
> I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.

And I'm saying that it is, with as much certainty as we can ever guess
the encoding of a file.

> If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal. ?It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

Sure, a Latin-1-encoded file could start with the same pattern that is
a UTF-8-encoded BOM. But at that point, a UTF-16-encoded file is also
valid Latin-1.

The question was in the context of encoding-guessing; if we're
guessing, a UTF-8-encoded BOM cannot signify anything else but UTF-8.
(Ditto for UTF-16 and UTF-32 BOMs.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:54:15 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:54:15 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100108T153305-228@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
Message-ID: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>
>> I wrote a new version of my patch (version 3):
>>
>> ?* don't change the default behaviour: use open(filename, encoding="BOM") to
>> check the BOM is there is any
>
> Well, I think if we implement this the default behaviour *should* be changed.
> It looks a bit senseless to have two different "auto-choose" options, one with
> encoding=None and one with encoding="BOM".

Well there *are* two different auto options: use the environment
variables (LANG etc.) or inspect the contents of the file. I think it
would be useful to have ways to specify both.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:56:46 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:56:46 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <ca471dc21001080756p5cb9f560x2a4443efa441fa7b@mail.gmail.com>

On Fri, Jan 8, 2010 at 1:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that. ?That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>>
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
>
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.
>
> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
>
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Sure. I said "crazy talk" only to stir up discussion. Which worked. :-)

Also, I don't want Python's default behavior to change -- sniffing the
encoding should be a separate option.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:59:45 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:59:45 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi6ibr$qag$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<hi6ibr$qag$1@ger.gmane.org>
Message-ID: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
> The BOM should not be seekeable if the file is opened with the proposed
> "guess encoding from BOM" mode: ?it isn't properly part of the stream at
> all in that case.

This feels about right to me. There are still questions though:
immediately after opening a file with a BOM, what should .tell()
return? And regardless of that, .seek(0) should put the file in that
same initial state.

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Fri Jan  8 17:03:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:03:07 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <loom.20100108T170053-283@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.




From mal at egenix.com  Fri Jan  8 17:25:22 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 17:25:22 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <4B475C72.1010207@egenix.com>

Guido van Rossum wrote:
> On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>>
>>> I wrote a new version of my patch (version 3):
>>>
>>>  * don't change the default behaviour: use open(filename, encoding="BOM") to
>>> check the BOM is there is any
>>
>> Well, I think if we implement this the default behaviour *should* be changed.
>> It looks a bit senseless to have two different "auto-choose" options, one with
>> encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Shouldn't this encoding guessing be a separate function that you call
on either a file or a seekable stream ?

After all, detecting encodings is just as useful to have for non-file
streams. You'd then avoid having to stuff everything into
a single function call and also open up the door for more complex
application specific guess work or defaults.

The whole process would then have two steps:

 1. guess encoding

  import codecs
  encoding = codecs.guess_file_encoding(filename)

 2. open the file with the found encoding

  f = open(filename, encoding=encoding)

For seekable streams f, you'd have:

 1. guess encoding

  import codecs
  encoding = codecs.guess_stream_encoding(f)

 2. wrap the stream with a reader for the found encoding

  reader_class = codecs.getreader(encoding)
  g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Jan  8 17:31:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:31:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <loom.20100108T171644-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver <at> palladion.com>
wrote:
> > The BOM should not be seekeable if the file is opened with the proposed
> > "guess encoding from BOM" mode:  it isn't properly part of the stream at
> > all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return?

tell() in the context of text I/O is specified to return an "opaque cookie". So
whatever value it returns would probably be fine, as long as seeking to that
value leaves the file in an acceptable state.

Rewinding (seeking to 0) in the presence of a BOM is already reasonably
supported by the TextIOWrapper object:

>>> dec = codecs.getincrementaldecoder('utf-16')()
>>> dec.decode(b'\xff\xfea\x00b\x00')
'ab'
>>> dec.decode(b'\xff\xfea\x00b\x00')
'\ufeffab'
>>> 
>>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
>>> f = io.TextIOWrapper(bio, encoding='utf-16')
>>> f.read()
'ab'
>>> f.seek(0)
0
>>> f.read()
'ab'

There are tests for this in test_io.py (test_encoded_writes, line 1929, and
test_append_bom and test_seek_bom, line 2045).

Regards

Antoine.




From python at mrabarnett.plus.com  Fri Jan  8 17:47:18 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 16:47:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081127.44044.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <4B476196.4080409@mrabarnett.plus.com>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
> (...)
>> (And yes, I know this happens. Doesn't mean we need to auto-guess by
>> default; there are lots of issues e.g. what should happen after
>> seeking to offset 0?)
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any
>  * fix for seek(0): always ignore the BOM
>  * add an unit test: check that the right encoding is detect, but also the the 
> BOM is ignored (especially after a seek(0))
> 
> BOM encoding doesn't work for writing into a file, so open(filename, "w", 
> encoding="BOM") raises a ValueError.
> 
I think it's similar to universal newline mode. You can tell it that
you're reading UTF-something-encoded text (common forms only).

The preference is UTF-8, but it could be UTF-8-sig (from Windows), or
possibly UTF-16/32, which really need a BOM because there are multiple
bytes per codepoint, so the bytes could be big-endian or little-endian.

The BOM (or signature) tells you what the encoding is, defaulting to
UTF-8 if there's none. If it subsequently raises a DecodeError, then
so be it!

Maybe there should also be a way of determining what encoding it decided
it was, so that you can then write a new file in that same encoding.


From status at bugs.python.org  Fri Jan  8 18:07:13 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jan 2010 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100108170714.014B078669@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/01/10 - 01/08/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2544 open (+27) / 16937 closed (+15) / 19481 total (+42)

Open issues with patches:  1017

Average duration of open issues: 708 days.
Median duration of open issues: 464 days.

Open Issues Breakdown
   open  2509 (+27)
pending    34 ( +0)

Issues Created Or Reopened (43)
_______________________________

Extended slicing with classic class behaves strangely            01/07/10
       http://bugs.python.org/issue7532    reopened mark.dickinson                
       patch                                                                   

optparse library documentation has an insignificant formatting i 01/01/10
CLOSED http://bugs.python.org/issue7618    created  vazovsky                      
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6           01/01/10
CLOSED http://bugs.python.org/issue7619    created  djc                           
                                                                               

Vim syntax highlight                                             01/02/10
       http://bugs.python.org/issue7620    created  july                          
       patch                                                                   

Test issue                                                       01/02/10
CLOSED http://bugs.python.org/issue7621    created  georg.brandl                  
                                                                               

[patch] improve unicode methods: split() rsplit() and replace()  01/03/10
       http://bugs.python.org/issue7622    created  flox                          
       patch                                                                   

PropertyType missing in Lib/types.py                             01/03/10
CLOSED http://bugs.python.org/issue7623    created  wplappert                     
                                                                               

isinstance(... ,collections.Callable) fails with oldstyle class  01/03/10
       http://bugs.python.org/issue7624    created  rgammans                      
                                                                               

bytearray needs more tests for "b.some_method()[0] is not b"     01/03/10
       http://bugs.python.org/issue7625    created  flox                          
       patch                                                                   

Entity references without semicolon in HTMLParser                01/03/10
CLOSED http://bugs.python.org/issue7626    created  stefan.schweizer              
                                                                               

mailbox.MH.remove() lock handling is broken                      01/04/10
       http://bugs.python.org/issue7627    created  sraustein                     
                                                                               

round() doesn't work correctly in 3.1.1                          01/04/10
CLOSED http://bugs.python.org/issue7628    created  bkovt                         
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs" 01/04/10
CLOSED http://bugs.python.org/issue7629    created  popelkopp                     
                                                                               

Strange behaviour of decimal.Decimal                             01/04/10
CLOSED http://bugs.python.org/issue7630    created  parmax                        
                                                                               

undefined label: bltin-file-objects                              01/04/10
CLOSED http://bugs.python.org/issue7631    created  ezio.melotti                  
                                                                               

dtoa.c: oversize b in quorem                                     01/04/10
       http://bugs.python.org/issue7632    created  skrah                         
                                                                               

decimal.py: type conversion in context methods                   01/04/10
       http://bugs.python.org/issue7633    created  skrah                         
       patch, easy                                                             

next/previous links in documentation skip some sections          01/05/10
CLOSED http://bugs.python.org/issue7634    created  gagenellina                   
                                                                               

19.6 xml.dom.pulldom doc: stub?                                  01/05/10
       http://bugs.python.org/issue7635    created  tjreedy                       
                                                                               

Add a set update action to optparse                              01/05/10
CLOSED http://bugs.python.org/issue7636    created  hardkrash                     
       patch                                                                   

Improve 19.5. xml.dom.minidom doc                                01/05/10
       http://bugs.python.org/issue7637    created  tjreedy                       
                                                                               

Counterintuitive str.splitlines() inconsistency.                 01/05/10
CLOSED http://bugs.python.org/issue7638    created  vencabot_teppoo               
                                                                               

bdist_msi fails on files with long names                         01/05/10
       http://bugs.python.org/issue7639    created  mmm77                         
                                                                               

buffered io seek() buggy                                         01/05/10
       http://bugs.python.org/issue7640    created  pakal                         
       patch                                                                   

Built-in Formatter accepts undocumented presentation type        01/06/10
CLOSED http://bugs.python.org/issue7641    created  lrekucki                      
                                                                               

[patch] Minor improvement in os.system doc                       01/06/10
       http://bugs.python.org/issue7642    created  Val                           
       patch                                                                   

What is an ASCII linebreak?                                      01/06/10
       http://bugs.python.org/issue7643    created  flox                          
                                                                               

bug in nntplib.body() method with possible fix                   01/06/10
       http://bugs.python.org/issue7644    created  mdmullins                     
       easy                                                                    

test_distutils fails on Windows XP                               01/06/10
       http://bugs.python.org/issue7645    created  austin987                     
                                                                               

test_telnetlib fails in Windows XP                               01/06/10
       http://bugs.python.org/issue7646    created  austin987                     
                                                                               

Add statvfs flags to the posix module                            01/06/10
       http://bugs.python.org/issue7647    created  ajax at redhat.com               
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                 01/06/10
       http://bugs.python.org/issue7648    created  austin987                     
       easy                                                                    

"u'%c' % char" broken for chars in range '\x80'-'\xFF'           01/07/10
       http://bugs.python.org/issue7649    created  ezio.melotti                  
       patch, easy                                                             

test_uuid is invalid                                             01/07/10
       http://bugs.python.org/issue7650    created  austin987                     
                                                                               

Python3: guess text file charset using the BOM                   01/07/10
       http://bugs.python.org/issue7651    created  haypo                         
       patch                                                                   

Merge C version of decimal into py3k.                            01/07/10
       http://bugs.python.org/issue7652    created  mark.dickinson                
                                                                               

Fix description of the way the PythonPath Windows registry key w 01/07/10
CLOSED http://bugs.python.org/issue7653    created  BoarGules                     
       patch, needs review                                                     

[patch] Enable additional bytes and memoryview tests.            01/07/10
       http://bugs.python.org/issue7654    created  flox                          
       patch                                                                   

PEP 374 Title usage & script redirection typo fixes              01/07/10
CLOSED http://bugs.python.org/issue7655    created  bab9e9                        
       patch                                                                   

test_hashlib fails on some installations (specifically Neal's re 01/08/10
       http://bugs.python.org/issue7656    created  r.david.murray                
                                                                               

test_ctypes failure on AIX 5.3                                   01/08/10
       http://bugs.python.org/issue7657    created  mallayya                      
       patch                                                                   

OS X pythonw.c compile error with 10.4 or earlier deployment tar 01/08/10
       http://bugs.python.org/issue7658    created  ned.deily                     
                                                                               

Problems with attribute assignment on object instances           01/08/10
       http://bugs.python.org/issue7659    created  pakal                         
                                                                               



Issues Now Closed (40)
______________________

shutil fails when copying to NTFS in Linux                        762 days
       http://bugs.python.org/issue1545    benjamin.peterson             
       patch                                                                   

Test                                                              751 days
       http://bugs.python.org/issue1619    georg.brandl                  
                                                                               

Windows Registry issue with 2.5.2 AMD64 msi                       645 days
       http://bugs.python.org/issue2539    loewis                        
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     618 days
       http://bugs.python.org/issue2698    Daniel26                      
                                                                               

optparse print_usage(),.. methods are not documented              542 days
       http://bugs.python.org/issue3340    ezio.melotti                  
                                                                               

reference leaks in test_distutils                                 500 days
       http://bugs.python.org/issue3660    ezio.melotti                  
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                          20 days
       http://bugs.python.org/issue3745    lemburg                       
       26backport                                                              

Add Option to Bind to a Local IP Address in httplib.py            464 days
       http://bugs.python.org/issue3972    gregory.p.smith               
       patch, easy                                                             

no reference for optparse methods                                 388 days
       http://bugs.python.org/issue4635    ezio.melotti                  
                                                                               

PyArg_Parse* should raise TypeError for float parsed with intege  339 days
       http://bugs.python.org/issue5080    mark.dickinson                
       patch                                                                   

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()              282 days
       http://bugs.python.org/issue5576    mark.dickinson                
       patch                                                                   

PyFrame_GetLineNumber                                             244 days
       http://bugs.python.org/issue5954    georg.brandl                  
       patch                                                                   

PyCode_NewEmpty                                                   244 days
       http://bugs.python.org/issue5959    georg.brandl                  
       patch                                                                   

Add non-command help topics to help completion of cmd.Cmd         241 days
       http://bugs.python.org/issue5991    georg.brandl                  
       patch                                                                   

make error                                                        231 days
       http://bugs.python.org/issue6067    georg.brandl                  
                                                                               

no longer possible to hash arrays                                 229 days
       http://bugs.python.org/issue6071    benjamin.peterson             
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           173 days
       http://bugs.python.org/issue6511    r.david.murray                
                                                                               

use different mechanism for pythonw on osx                        127 days
       http://bugs.python.org/issue6834    ned.deily                     
       easy                                                                    

Py3k doc: "from __future__ import division" not necessary          32 days
       http://bugs.python.org/issue7432    ezio.melotti                  
                                                                               

cPickle: stack underflow in load_pop()                             31 days
       http://bugs.python.org/issue7455    pitrou                        
       patch                                                                   

crash in str.rfind() with an invalid start value                   25 days
       http://bugs.python.org/issue7458    pitrou                        
       patch                                                                   

Implement fastsearch algorithm for rfind/rindex                    26 days
       http://bugs.python.org/issue7462    ezio.melotti                  
       patch                                                                   

GZipFile.readline too slow                                         20 days
       http://bugs.python.org/issue7471    pitrou                        
       patch                                                                   

ssl module documentation: SSLSocket.unwrap description shown twi    5 days
       http://bugs.python.org/issue7592    georg.brandl                  
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            7 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

optparse library documentation has an insignificant formatting i    2 days
       http://bugs.python.org/issue7618    ezio.melotti                  
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6              1 days
       http://bugs.python.org/issue7619    djc                           
                                                                               

Test issue                                                          0 days
       http://bugs.python.org/issue7621    ezio.melotti                  
                                                                               

PropertyType missing in Lib/types.py                                0 days
       http://bugs.python.org/issue7623    benjamin.peterson             
                                                                               

Entity references without semicolon in HTMLParser                   2 days
       http://bugs.python.org/issue7626    r.david.murray                
                                                                               

round() doesn't work correctly in 3.1.1                             0 days
       http://bugs.python.org/issue7628    benjamin.peterson             
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs"    0 days
       http://bugs.python.org/issue7629    r.david.murray                
                                                                               

Strange behaviour of decimal.Decimal                                0 days
       http://bugs.python.org/issue7630    mark.dickinson                
                                                                               

undefined label: bltin-file-objects                                 0 days
       http://bugs.python.org/issue7631    pitrou                        
                                                                               

next/previous links in documentation skip some sections             0 days
       http://bugs.python.org/issue7634    georg.brandl                  
                                                                               

Add a set update action to optparse                                 3 days
       http://bugs.python.org/issue7636    r.david.murray                
       patch                                                                   

Counterintuitive str.splitlines() inconsistency.                    0 days
       http://bugs.python.org/issue7638    flox                          
                                                                               

Built-in Formatter accepts undocumented presentation type           0 days
       http://bugs.python.org/issue7641    eric.smith                    
                                                                               

Fix description of the way the PythonPath Windows registry key w    0 days
       http://bugs.python.org/issue7653    r.david.murray                
       patch, needs review                                                     

PEP 374 Title usage & script redirection typo fixes                 0 days
       http://bugs.python.org/issue7655    georg.brandl                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 22 [patch] improve unicode methods: split() rsplit() and replace()    5 days
open    http://bugs.python.org/issue7622   

 14 Test suite emits many DeprecationWarnings when -3 is enabled      91 days
open    http://bugs.python.org/issue7092   

 13 unicode_escape codec does not escape quotes                        8 days
open    http://bugs.python.org/issue7615   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    0 days
open    http://bugs.python.org/issue7658   

  8 Merge C version of decimal into py3k.                              1 days
open    http://bugs.python.org/issue7652   

  8 "u'%c' % char" broken for chars in range '\x80'-'\xFF'             2 days
open    http://bugs.python.org/issue7649   

  8 decimal.py: type conversion in context methods                     4 days
open    http://bugs.python.org/issue7633   

  8 Patch for [ 735515 ] urllib2 should cache 301 redir              906 days
open    http://bugs.python.org/issue1755841

  7 Test issue                                                         0 days
closed  http://bugs.python.org/issue7621   

  7 Cannot use both read and readline method in same ZipExtFile obj    8 days
open    http://bugs.python.org/issue7610   




From tseaver at palladion.com  Fri Jan  8 22:09:54 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:09:54 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <hi86v3$3j5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> The BOM should not be seekeable if the file is opened with the proposed
>> "guess encoding from BOM" mode:  it isn't properly part of the stream at
>> all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return? And regardless of that, .seek(0) should put the file in that
> same initial state.

I think the behavior should be something like:

 >>> f = open('/path/to/maybe-BOM-encoded-file', 'r', encoding='BOM')
 >>> f.tell()
 0L
 >>> f.seek(-1)
 >>> f.tell() # count of unicode chars in decoded stream
 45L
 >>> f.seek(0)
 >>> f.read(1) # read first unicode char decoded from stream.
 'A'

In other words, the BOM is not readable / seekable at all:  it is
invisible to the consumer of the decoded stream.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHnyIACgkQ+gerLs4ltQ6s3QCgznD+7FbUzfCbe5TS6OcoXjMg
rdgAoJAMEXe2xwLCIwJaZ6XA6rVyTIAi
=oXb3
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:19:10 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:19:10 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B475C72.1010207@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com>
Message-ID: <hi87gg$8ge$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:

> Shouldn't this encoding guessing be a separate function that you call
> on either a file or a seekable stream ?
> 
> After all, detecting encodings is just as useful to have for non-file
> streams.

Other stream sources typically have out-of-band ways to signal the
encoding:  only when reading from the filesystem do we pretty much
*have* to guess, and in that case the BOM / signature is the best
heuristic we have.  Also, some non-file streams are not seekable, and so
can't be guessed via a pre-pass.

> You'd then avoid having to stuff everything into
> a single function call and also open up the door for more complex
> application specific guess work or defaults.
> 
> The whole process would then have two steps:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_file_encoding(filename)

Filename is not enough information:  or do you mean that API to actually
open the stream?

>  2. open the file with the found encoding
> 
>   f = open(filename, encoding=encoding)
> 
> For seekable streams f, you'd have:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_stream_encoding(f)
> 
>  2. wrap the stream with a reader for the found encoding
> 
>   reader_class = codecs.getreader(encoding)
>   g = reader_class(f)
> 


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoU4ACgkQ+gerLs4ltQ5o3QCeLOJ7J91E+5f66vhgu1BUhYh4
9UgAnR2IeCd0BCsPez8ZilGNHJfhRn3Y
=SoPb
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:14:59 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:14:59 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <hi878l$7os$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:

>>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that.  That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
> 
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.

If the programmer opens the file using a "guess using the BOM" encoding,
 Python should *not* attempt to verify that the file is properly
encoded:  it should check for (and consume) any BOM, and then return a
stream which uses the encoding inferred from the BOM.  Any errors should
be handled later, when characters are read, exactly as if the file had
been opened with the same encoding guessed from the BOM.

> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
> 
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Agreed.  Having that marker at the start of the file makes interop with
other tools *much* easier.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoFMACgkQ+gerLs4ltQ73dACffwUfyh6Q9vUnKYf367QFjNcU
RRMAoNuKCWEx7j+MSdTv+UjhAPynBc14
=uAX6
-----END PGP SIGNATURE-----



From eric at trueblade.com  Fri Jan  8 22:40:47 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 8 Jan 2010 16:40:47 -0500 (EST)
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
Message-ID: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>

>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
>>
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
>
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

But what if the file were in (for example) a zip file? I think you
definitely want to have access to this functionality outside of open().

Eric.



From foom at fuhm.net  Fri Jan  8 22:49:23 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 8 Jan 2010 16:49:23 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de> <hi878l$7os$1@ger.gmane.org>
Message-ID: <D481E750-3EE7-4498-9959-B4E34E02FFC6@fuhm.net>

On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote:
>> I understood this proposal as a general processing guideline, not
>> something the io library should do (but, say, a text editor).
>>
>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly  
>> have
>> a byte order - hence I call it a signature, not the BOM. As a  
>> signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote  
>> the
>> encoding is useful and helps reducing moji-bake. This is the reason  
>> it's
>> used on Windows: notepad would normally assume that text is in the  
>> ANSI
>> code page, and for compatibility, it can't stop doing that. So the  
>> UTF-8
>> signature gives them an exit strategy.
>
> Agreed.  Having that marker at the start of the file makes interop  
> with
> other tools *much* easier.

Putting the BOM at the beginning of UTF-8 text files is not a good  
idea, it makes interop much *worse* on a unix system, not better.  
Without the BOM, most commands do the right thing with UTF-8 text.  
E.g. to concatenate two files:

$ cat file-1 file-2 > file-3

With a BOM at the beginning of the file, it won't work right. Of  
course, you could modify "cat" (and every other stream processing  
command) to know how to consume and emit BOMs, and omit the extra one  
that would show up in the middle of the stream...but even that can't  
work; what about:
$ (cat file-1; cat file-2) > file-3.

Should the shell now know that when you run multiple commands, it  
should eat the BOM emitted from the second command?

Basically, using a BOM in a utf-8 file is just not a good idea: it  
completely ruins interop with every standard unix tool.

This is not to say that Python shouldn't have a way to read a file  
with a UTF-8 BOM: it just shouldn't encourage you to *write* such files.

James


From mal at egenix.com  Fri Jan  8 22:51:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 22:51:26 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>	<4B475C72.1010207@egenix.com>
	<hi87gg$8ge$1@ger.gmane.org>
Message-ID: <4B47A8DE.1080401@egenix.com>

Tres Seaver wrote:
> M.-A. Lemburg wrote:
> 
>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
> 
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
> 
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

Sure there are non-seekable file streams, but at least when
using StringIO-type streams you don't have that restriction.

An encoding detection function would provide more use in other
cases as well, so instead of hiding away the functionality in
the open() constructor, I'm suggesting to make expose it via
the codecs module.

Applications can then use it where necessary and also provide their
own defaults, using other heuristics.

>> You'd then avoid having to stuff everything into
>> a single function call and also open up the door for more complex
>> application specific guess work or defaults.
> 
>> The whole process would then have two steps:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_file_encoding(filename)
> 
> Filename is not enough information:  or do you mean that API to actually
> open the stream?

Yes. The API would open the file, guess the encoding and then
close it again. If you don't want that to happen, you could use
the second API I mentioned below on the already open file.

Note that this function could detect a lot more encodings with
reasonably high probability than just BOM-prefixed ones,
e.g. we could also add support to detect encoding declarations
such as the ones we use in Python source files.

>>  2. open the file with the found encoding
> 
>>   f = open(filename, encoding=encoding)
> 
>> For seekable streams f, you'd have:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_stream_encoding(f)

I forgot to mention: This API needs to position the file pointer
to the start of the first data byte.

>>  2. wrap the stream with a reader for the found encoding
> 
>>   reader_class = codecs.getreader(encoding)
>>   g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From tseaver at palladion.com  Fri Jan  8 22:59:04 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:59:04 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <hi89r8$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
>>> Shouldn't this encoding guessing be a separate function that you call
>>> on either a file or a seekable stream ?
>>>
>>> After all, detecting encodings is just as useful to have for non-file
>>> streams.
>> Other stream sources typically have out-of-band ways to signal the
>> encoding:  only when reading from the filesystem do we pretty much
>> *have* to guess, and in that case the BOM / signature is the best
>> heuristic we have.  Also, some non-file streams are not seekable, and so
>> can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

If the application expects a possibly-BOM-signature-marked file, but you
pass it mismatched garbage:

  >>> f = open('some.zip', encoding='BOM")

the error handling should be the same as if you passed any other
mismatched encoding:

  >>> f = open('some.zip', encoding='UTF8')

i.e., you discover the error when you try to read from the (non)encoded
stream, not when you open it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHqpwACgkQ+gerLs4ltQ7uAACeKEc+WT4TASGcVl1Hfqe6L9La
I6EAn1pJtngtLWPdothGbYB+zUabEvTW
=TjBK
-----END PGP SIGNATURE-----



From victor.stinner at haypocalc.com  Fri Jan  8 23:10:32 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 23:10:32 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <201001082310.33029.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 22:40:47, Eric Smith a ?crit :
> >> Shouldn't this encoding guessing be a separate function that you call
> >> on either a file or a seekable stream ?
> >>
> >> After all, detecting encodings is just as useful to have for non-file
> >> streams.
> >
> > Other stream sources typically have out-of-band ways to signal the
> > encoding:  only when reading from the filesystem do we pretty much
> > *have* to guess, and in that case the BOM / signature is the best
> > heuristic we have.  Also, some non-file streams are not seekable, and so
> > can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

FYI my patch (encoding="BOM") is implemented in TextIOWrapper, and 
TextIOWrapper takes a binary stream as input, not a filename.

-- 
Victor Stinner
http://www.haypocalc.com/


From yoann.padioleau at facebook.com  Fri Jan  8 23:59:52 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Fri, 8 Jan 2010 14:59:52 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464F1C.7020404@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
	<4B464F1C.7020404@v.loewis.de>
Message-ID: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>


On Jan 7, 2010, at 1:16 PM, Martin v. L?wis wrote:

>>> astgen.py is not used to process asdl files; ast.txt lives right
>>> next to astgen.py. Instead, the asdl file is processed by
>>> Parser/asdl_c.py.
>> 
>> Yes, I know that. That's why I asked about the relation between
>> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
>> expose as a reflection mechanism things that were generated from
>> ast.txt, then there could be a mismatch. Where does ast.txt comes
>> from ? Shouldn't it be generated itself from Python.adsl ?
> 
> What you may not be aware of is that Tools/compiler (and the
> compiler package that it builds on) are both unused and unmaintained.

I see. So if people want to analyze python code they have to use
other tools (like rope?) ? or use reflection ?

> 
> If the package stops working correctly - tough luck.
> 
>> So we would have
>> 
>> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
>> containing all the UnarySub, Expression, classes that represents a
>> Python AST.
> 
> No - what actually happens in Python 3.x is this: both the compiler
> package and Tools/compiler are removed.

Ok. I will then create my own ast classes generator.

Thanks.


> 
> Regards,
> Martin



From g.brandl at gmx.net  Sat Jan  9 00:10:24 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 09 Jan 2010 00:10:24 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<4B46F54D.9090103@v.loewis.de>
	<hi878l$7os$1@ger.gmane.org>
Message-ID: <hi8e1n$sos$1@ger.gmane.org>

Am 08.01.2010 22:14, schrieb Tres Seaver:

>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly have
>> a byte order - hence I call it a signature, not the BOM. As a signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote the
>> encoding is useful and helps reducing moji-bake. This is the reason it's
>> used on Windows: notepad would normally assume that text is in the ANSI
>> code page, and for compatibility, it can't stop doing that. So the UTF-8
>> signature gives them an exit strategy.
> 
> Agreed.  Having that marker at the start of the file makes interop with
> other tools *much* easier.

Except if only 50% of the other tools support the signature.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From martin at v.loewis.de  Sat Jan  9 00:57:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 00:57:46 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>	<4B464499.4020305@v.loewis.de>	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>	<4B464F1C.7020404@v.loewis.de>
	<F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
Message-ID: <4B47C67A.3020302@v.loewis.de>

> I see. So if people want to analyze python code they have to use
> other tools (like rope?) ? or use reflection ?

Correct. One such tool might be the true Python compiler, along
with the _ast module.

Regards,
Martin


From victor.stinner at haypocalc.com  Sat Jan  9 00:59:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 00:59:00 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
Message-ID: <201001090059.00848.victor.stinner@haypocalc.com>

Hi,

Thanks for all the answers! I will try to sum up all ideas here.


(1) Change default open() behaviour or make it optional?

Guido would like to add an option and keep open() unchanged. He wrote that 
checking for BOM and using system locale are too much different to be the same 
option (encoding=None).

Antoine would like to check BOM by default, because both options (system 
locale vs checking for BOM) is the same thing.

About Antoine choice (encoding=None): which file modes would check for a BOM? 
I would like to answer only the read only mode, but then open(filename, "r") 
and open(filename, "r+") would behave differently?

  => 1 point for Guido (encoding="BOM" is more explicit)

Antoine choice has the advantage of directly support UTF-8+BOM, UTF-16 and 
UTF-32 for all applications and all modules using open(filename).

  => 1 point for Antoine


(2) Check for a BOM while reading or detect it before?

Everybody agree that checking BOM is an interesting option and should not be 
limited to open().

Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
name or a binary file-like object: it returns the encoding and seek to the 
file start or just after the BOM.

I dislike this function because it requires extra file operations (open 
(optional), read() and seek()) and it doesn't work if the file is not seekable 
(eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
avoid extra file operations.

Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
any file-like object.


(3) tell() and seek() on a text file starting with a BOM

To be consistent with Antoine example:

   >>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
   >>> f = io.TextIOWrapper(bio, encoding='utf-16')
   >>> f.read()
   'ab'
   >>> f.seek(0)
   0
   >>> f.read()
   'ab'

TextIOWrapper:

 * tell() should return zero at file start,
 * seek(0) should go be to file start,
 * and the BOM should always be "ignored".

I mean:

  with open("utf8bom.txt", encoding="BOM") as fp:
     assert fp.tell() == 0   
     text = fp.read() # no BOM here
     fp.seek(0)
     assert fp.read() == text

--

About my patch:

 - BOM check is explicit: open(filebame,  encoding="BOM")
 - tell() / seek(0) works as expected
 - BOM bytes are always skipped in read() / readlines() result

Hum, I don't know if this email can be called a sum up ;-)

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 01:09:48 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 00:09:48 +0000 (UTC)
Subject: [Python-Dev] Quick sum up about open() + BOM
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <loom.20100109T010657-648@post.gmane.org>

Hello Victor,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> (1) Change default open() behaviour or make it optional?
> 
[...]
> 
> Antoine would like to check BOM by default, because both options (system 
> locale vs checking for BOM) is the same thing.

To be clear, I am not saying it is the same thing. What I think is that it would
be a mistake to use a mildly unreliable heuristic by default (the locale +
device encoding heuristic) but refuse to trust a more reliable heuristic (the
BOM-based detection algorithm).

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Sat Jan  9 01:14:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 00:14:39 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <loom.20100109T010657-648@post.gmane.org>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<loom.20100109T010657-648@post.gmane.org>
Message-ID: <4B47CA6F.5000607@voidspace.org.uk>

On 09/01/2010 00:09, Antoine Pitrou wrote:
> Hello Victor,
>
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>    
>> (1) Change default open() behaviour or make it optional?
>>
>>      
> [...]
>    
>> Antoine would like to check BOM by default, because both options (system
>> locale vs checking for BOM) is the same thing.
>>      
> To be clear, I am not saying it is the same thing. What I think is that it would
> be a mistake to use a mildly unreliable heuristic by default (the locale +
> device encoding heuristic) but refuse to trust a more reliable heuristic (the
> BOM-based detection algorithm).
>    

I concur. On Windows both UTF-8 and signature are very common, yet the 
platform default is the truly awful CP1252.

All the best,

Michael
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From floris.bruynooghe at gmail.com  Sat Jan  9 01:26:05 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 9 Jan 2010 00:26:05 +0000
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B46F6D7.1050301@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
	<4B46F6D7.1050301@v.loewis.de>
Message-ID: <20100109002605.GB13536@laurie.devork>

On Fri, Jan 08, 2010 at 10:11:51AM +0100, "Martin v. L?wis" wrote:
> Nicholas Bastin wrote:
> > I think this problem probably needs to move over to distutils-sig, as
> > it doesn't seem to be specific to the way that Python itself uses
> > distutils.
> 
> I'm fairly skeptical that anybody on distutils SIG is interested in
> details of the Python build process...

Uh, hum.  Unfounded skepticism.  ;-)
But as said filing a bug sounds better in this case.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From v+python at g.nevcal.com  Sat Jan  9 01:47:38 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 16:47:38 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B47D22A.8070305@g.nevcal.com>

On approximately 1/8/2010 3:59 PM, came the following characters from 
the keyboard of Victor Stinner:
> Hi,
>
> Thanks for all the answers! I will try to sum up all ideas here.

One concern I have with this implementation encoding="BOM" is that if 
there is no BOM it assumes UTF-8.  That is probably a good assumption in 
some circumstances, but not in others.

* It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
(cases where the endianness is unspecified) contain a BOM.  Hence, it 
might be that someone would expect a UTF-16LE (or any of the formats 
that don't require a BOM, rather than UTF-8), but be willing to accept 
any BOM-discriminated format.

* Potentially, this could be expanded beyond the various Unicode 
encodings... one could envision that a program whose data files 
historically were in any particular national language locale, could want 
to be enhance to accept Unicode, and could declare that they will accept 
any BOM-discriminated format, but want to default, in the absence of a 
BOM, to the original national language locale that they historically 
accepted.  That would provide a migration path for their old data files.

So the point is, that it might be nice to have 
"BOM-otherEncodingForDefault" for each other encoding that Python 
supports.  Not sure that is the right API, but I think it is expressive 
enough to handle the cases above.  Whether the cases solve actual 
problems or not, I couldn't say, but they seem like reasonable cases.

It would, of course, be nicest if OS metadata had been invented way back 
when, for all OSes, such that all text files were flagged with their 
encoding... then languages could just read the encoding and do the right 
thing! But we live in the real world, instead.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From python at mrabarnett.plus.com  Sat Jan  9 02:12:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 09 Jan 2010 01:12:28 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B47D7FC.4070703@mrabarnett.plus.com>

Glenn Linderman wrote:
> On approximately 1/8/2010 3:59 PM, came the following characters from 
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if 
> there is no BOM it assumes UTF-8.  That is probably a good assumption in 
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
> (cases where the endianness is unspecified) contain a BOM.  Hence, it 
> might be that someone would expect a UTF-16LE (or any of the formats 
> that don't require a BOM, rather than UTF-8), but be willing to accept 
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode 
> encodings... one could envision that a program whose data files 
> historically were in any particular national language locale, could want 
> to be enhance to accept Unicode, and could declare that they will accept 
> any BOM-discriminated format, but want to default, in the absence of a 
> BOM, to the original national language locale that they historically 
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have 
> "BOM-otherEncodingForDefault" for each other encoding that Python 
> supports.  Not sure that is the right API, but I think it is expressive 
> enough to handle the cases above.  Whether the cases solve actual 
> problems or not, I couldn't say, but they seem like reasonable cases.
> 
> It would, of course, be nicest if OS metadata had been invented way back 
> when, for all OSes, such that all text files were flagged with their 
> encoding... then languages could just read the encoding and do the right 
> thing! But we live in the real world, instead.
> 
What about listing the possible encodings? It would try each in turn
until it found one where the BOM matched or had no BOM:

     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')

or is that taking it too far?


From martin at v.loewis.de  Sat Jan  9 02:23:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 02:23:07 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47CA6F.5000607@voidspace.org.uk>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<loom.20100109T010657-648@post.gmane.org>
	<4B47CA6F.5000607@voidspace.org.uk>
Message-ID: <4B47DA7B.6050505@v.loewis.de>

>>> Antoine would like to check BOM by default, because both options
>>> (system locale vs checking for BOM) is the same thing.
>>> 
>> To be clear, I am not saying it is the same thing. What I think is 
>> that it would be a mistake to use a mildly unreliable heuristic by
>> default (the locale + device encoding heuristic) but refuse to
>> trust a more reliable heuristic (the BOM-based detection
>> algorithm).
>> 
> 
> I concur. On Windows both UTF-8 and signature are very common, yet
> the platform default is the truly awful CP1252.

While I would support combining BOM detection in the case where a file
is opened for reading and no encoding is specified, I see two problems:
a) if a seek operations is performed before having looked at the BOM,
   no determination would have been made
b) what encoding should it use on writing?

Regards,
Martin



From v+python at g.nevcal.com  Sat Jan  9 02:49:12 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 17:49:12 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <4B47E098.6030703@g.nevcal.com>

On approximately 1/8/2010 5:12 PM, came the following characters from 
the keyboard of MRAB:
> Glenn Linderman wrote:
>> On approximately 1/8/2010 3:59 PM, came the following characters from 
>> the keyboard of Victor Stinner:
>>> Hi,
>>>
>>> Thanks for all the answers! I will try to sum up all ideas here.
>>
>> One concern I have with this implementation encoding="BOM" is that if 
>> there is no BOM it assumes UTF-8.  That is probably a good assumption 
>> in some circumstances, but not in others.
>>
>> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
>> encoded files include a BOM.  It is only required that UTF-16 and 
>> UTF-32 (cases where the endianness is unspecified) contain a BOM.  
>> Hence, it might be that someone would expect a UTF-16LE (or any of 
>> the formats that don't require a BOM, rather than UTF-8), but be 
>> willing to accept any BOM-discriminated format.
>>
>> * Potentially, this could be expanded beyond the various Unicode 
>> encodings... one could envision that a program whose data files 
>> historically were in any particular national language locale, could 
>> want to be enhance to accept Unicode, and could declare that they 
>> will accept any BOM-discriminated format, but want to default, in the 
>> absence of a BOM, to the original national language locale that they 
>> historically accepted.  That would provide a migration path for their 
>> old data files.
>>
>> So the point is, that it might be nice to have 
>> "BOM-otherEncodingForDefault" for each other encoding that Python 
>> supports.  Not sure that is the right API, but I think it is 
>> expressive enough to handle the cases above.  Whether the cases solve 
>> actual problems or not, I couldn't say, but they seem like reasonable 
>> cases.
>>
>> It would, of course, be nicest if OS metadata had been invented way 
>> back when, for all OSes, such that all text files were flagged with 
>> their encoding... then languages could just read the encoding and do 
>> the right thing! But we live in the real world, instead.
>>
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
>
>     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

That sounds very flexible -- but in net effect it would only make 
illegal a subset of the BOM-containing encodings (those not listed) 
without making legal any additional encodings other than the non-BOM 
encoding.  Whether prohibiting a subset of BOM-containing encodings is a 
useful use case, I couldn't say... but my goal would be to included as 
many different file encodings on input as possible: without a BOM, that 
is exactly 1 (unless there are other heuristics), with a BOM, it is 
1+all-BOM-containing encodings.  Your scheme would permit numbers of 
encodings accepted to vary between 1 and 1+all-BOM-containing encodings.

(I think everyone can agree there are 5 different byte sequences that 
can be called a Unicode BOM.  The likelihood of them appearing in any 
other text encoding created by mankind depends on those other encodings 
-- but it is not impossible.  It is truly up to the application to 
decide whether BOM detection could potentially conflict with files in 
some other encoding that would be acceptable to the application.)

So I think it is taking it further than I can see value in, but I'm 
willing to be convinced otherwise.  I see only a need for detecting BOM, 
and specifying a default encoding to be used if there is no BOM.  Note 
that it might be nice to have a specification for using current 
encoding=None heuristic -- perhaps encoding="BOM-None" per my originally 
proposed syntax.  But I'm still not saying that is the best syntax.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From ncoghlan at gmail.com  Sat Jan  9 04:07:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Jan 2010 13:07:12 +1000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B476196.4080409@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>
	<4B476196.4080409@mrabarnett.plus.com>
Message-ID: <4B47F2E0.7090403@gmail.com>

MRAB wrote:
> Maybe there should also be a way of determining what encoding it decided
> it was, so that you can then write a new file in that same encoding.

I thought of that question as well - the f.encoding attribute on the
opened file should be sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Sat Jan  9 06:48:36 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 06:48:36 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47E098.6030703@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com> <4B47D7FC.4070703@mrabarnett.plus.com>
	<4B47E098.6030703@g.nevcal.com>
Message-ID: <319e029f1001082148h5cee2c0bn76f70a17766ca69e@mail.gmail.com>

It seems to me that when opening a file, the following is the only
flow that makes sense for the typical opening of a file flow:

if encoding is not None:
   use encoding
elif file has BOM:
   use BOM
else:
   use system default

And hence a encoding='BOM' isn't needed there. Although I'm trying to
come up with usecases that doesn't work with this, I can't. :)

BUT

When writing things are not so easy though. Apparently some encodings
require a BOM to be written, but others do not, but allow it, and some
has no byte order mark. So there you have to be able to write the BOM,
or not. And that's either a new parameter, because you can't use
encoding='BOM' since you need to specify the encoding as well, or a
new method.

I would suggest a BOM parameter, and maybe a method as  well.

BOM=None|True|False

Where "None" means a sane default behaviour, that is write a BOM if
the encoding require it.
"True" means write a BOM if the encoding *supports* it.
"False" means Don't write a BOM even if the encoding requires it
(because I know what I'm doing)

if 'w' in mode: # But not 'r' or 'a'
    if BOM == True and encoding in (ENCODINGS THAT ALLOW BOM):
        write_bom = True
    elif BOM == False:
       write_bom = False
    elif BOM == None and encoding in (ENCODINGS THAT REQUIRE BOM):
          write_bom = True
    else:
          write_bom = False
else:
    write_bom = False

For reading this parameter could either be a noop, or possibly change
the behavior somehow, if a usecase where that makes sense can be
imagined.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From walter at livinglogic.de  Sat Jan  9 11:51:57 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 11:51:57 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B485FCD.2040901@livinglogic.de>

On 09.01.10 01:47, Glenn Linderman wrote:

> On approximately 1/8/2010 3:59 PM, came the following characters from
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.  That is probably a good assumption in
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32
> (cases where the endianness is unspecified) contain a BOM.  Hence, it
> might be that someone would expect a UTF-16LE (or any of the formats
> that don't require a BOM, rather than UTF-8), but be willing to accept
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode
> encodings... one could envision that a program whose data files
> historically were in any particular national language locale, could want
> to be enhance to accept Unicode, and could declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have
> "BOM-otherEncodingForDefault" for each other encoding that Python
> supports.  Not sure that is the right API, but I think it is expressive
> enough to handle the cases above.  Whether the cases solve actual
> problems or not, I couldn't say, but they seem like reasonable cases.

This is doable with the currect API. Simply define a codec search
function that handles all encoding names that start with "BOM-" and pass
the "otherEncodingForDefault" part along to the codec.

> It would, of course, be nicest if OS metadata had been invented way back
> when, for all OSes, such that all text files were flagged with their
> encoding... then languages could just read the encoding and do the right
> thing! But we live in the real world, instead.

Servus,
   Walter


From walter at livinglogic.de  Sat Jan  9 12:18:33 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 12:18:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081140.28124.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
Message-ID: <4B486609.2050804@livinglogic.de>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
>>> Builtin open() function is unable to open an UTF-16/32 file starting with
>>> a BOM if the encoding is not specified (raise an unicode error). For an
>>> UTF-8 file starting with a BOM, read()/readline() returns also the BOM
>>> whereas the BOM should be "ignored".
>> It depends. If you use the utf-8-sig encoding, it *will* ignore the
>> UTF-8 signature.
> 
> Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
> UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
> remove the BOM after the first read (much harder if you use a module like 
> ConfigParser or csv).
> 
>>> Since my proposition changes the result TextIOWrapper.read()/readline()
>>> for files starting with a BOM, we might introduce an option to open() to
>>> enable the new behaviour. But is it really needed to keep the backward
>>> compatibility?
>> Absolutely. And there is no need to produce a new option, but instead
>> use the existing options: define an encoding that auto-detects the
>> encoding from the family of BOMs. Maybe you call it encoding="sniff".
> 
> Good idea, I choosed open(filename, encoding="BOM").

On the surface this looks like there's an encoding named "BOM", but 
looking at your patch I found that the check is still done in 
TextIOWrapper. IMHO the best approach would to the implement a *real* 
codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
the IO library. It could even be developed as a standalone project and 
published in the Cheeseshop.

To see how something like this can be done, take a look at the UTF-16 
codec, that switches to bigendian or littleendian mode depending on the 
first read/decode call.

Servus,
    Walter







From victor.stinner at haypocalc.com  Sat Jan  9 13:37:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:37:06 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47DA7B.6050505@v.loewis.de>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47CA6F.5000607@voidspace.org.uk> <4B47DA7B.6050505@v.loewis.de>
Message-ID: <201001091337.06596.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:23:07, Martin v. L?wis a ?crit :
> While I would support combining BOM detection in the case where a file
> is opened for reading and no encoding is specified, I see two problems:
> a) if a seek operations is performed before having looked at the BOM,
>    no determination would have been made

TextIOWrapper doesn't support seek to an arbitrary byte. It uses "cookie" 
which is an opaque value. Reuse a cookie from another file or an old cookie is 
forbidden (but it doesn't raise an error). This is not specific to the BOM 
checking: the problem already exist for encodings using a BOM (eg. UTF-16).

> b) what encoding should it use on writing?

Don't change anything to writing.

With Antoince choice: open('file.txt', 'w', encoding=None) continue to use the 
actual heuristic (os.device_encoding() or system locale).

With Guido choice, encoding="BOM": it raises an error, because BOM check is 
not supported when writing into a file. How could the BOM be checked when 
creating a new (empty) file!?

-- 
Victor Stinner
http://www.haypocalc.com/


From mal at egenix.com  Sat Jan  9 13:45:58 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 09 Jan 2010 13:45:58 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B487A86.4020603@egenix.com>

Victor Stinner wrote:
> (2) Check for a BOM while reading or detect it before?
> 
> Everybody agree that checking BOM is an interesting option and should not be 
> limited to open().
> 
> Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
> name or a binary file-like object: it returns the encoding and seek to the 
> file start or just after the BOM.
> 
> I dislike this function because it requires extra file operations (open 
> (optional), read() and seek()) and it doesn't work if the file is not seekable 
> (eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
> avoid extra file operations.
> 
> Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
> any file-like object.

Yes, but the implementation is limited to just BOM checking
and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

With a codecs module function we could easily extend the
encoding detection to more file types, e.g. XML files,
Python source code files, etc. that use other mechanisms
for defining the encoding.

BTW: I haven't looked at your implementation, but what happens
when your BOM check fails ? Will the implementation add the
already read bytes back to a buffer ?

This rollback action is the only reason for needing a
seekable stream in codecs.guess_stream_encoding().

Another point to consider:

AFAIK, we currently have a moratorium on changes to Python
builtins. How does that match up with the proposed changes ?

Using a new codec like Walter suggested would move the
implementation into the stdlib for which doesn't the
moratorium doesn't apply.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From victor.stinner at haypocalc.com  Sat Jan  9 13:54:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:54:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <201001091354.17239.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 01:47:38, vous avez ?crit :
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.

If no BOM is found, it fallback to the current heuristic: os.device_encoding() 
or system local.

> (...) Hence, it might be that someone would expect a UTF-16LE (or any of 
> the formats that don't require a BOM, rather than UTF-8), but be willing 
> to accept any BOM-discriminated format.
> (...) declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted

You mean "if there is a BOM, use it, otherwise fallback to a specific 
charset"? How could it be declared? Maybe:

   open("file.txt", check_bom=True, encoding="UTF16-LE")
   open("file.txt", check_bom=True, encoding="latin1")

About falling back to UTF-8, it would be written:

   open("file.txt", check_bom=True, encoding="UTF-8")

As explained before, check_bom=True is only accepted for read only file mode.

Well, why not. This is a third choice for my point (1) :-) It's between Guido 
and Antoine choice, and I like it because we can fallback to UTF-8 instead of 
the dummy system locale: Windows users will be happy to be able to use UTF-8 
:-) I prefer to fallback to a fixed encoding then depending on the system 
locale.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:34:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:34:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <201001091434.17521.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:12:28, MRAB a ?crit :
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
> 
>      my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

Yes, you're taking it foo far :-) Checking BOM is reliable, whereas *guessing* 
the charset only using the byte stream can only be an heuristic. Guess a 
charset is a complex problem, they are 3rd party library to do that, like the 
chardet project.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:38:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:38:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B486609.2050804@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <201001091438.43576.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
> > Good idea, I choosed open(filename, encoding="BOM").
> 
> On the surface this looks like there's an encoding named "BOM", but
> looking at your patch I found that the check is still done in
> TextIOWrapper. IMHO the best approach would to the implement a *real*
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
> the IO library. It could even be developed as a standalone project and
> published in the Cheeseshop.

Why not, this is another solution to the point (2) (Check for a BOM while 
reading or detect it before?). Which encoding would be used if there is not 
BOM? UTF-8 sounds like a good choice.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:50:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:50:28 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B487A86.4020603@egenix.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B487A86.4020603@egenix.com>
Message-ID: <201001091450.28497.victor.stinner@haypocalc.com>

Hi,

Le samedi 09 janvier 2010 13:45:58, vous avez ?crit :
> > Note: I implemented the BOM check in TextIOWrapper; so it's already
> > usable for any file-like object.
> 
> Yes, but the implementation is limited to just BOM checking
> and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

Sure, but that's already better than no BOM check :-) It looks like many 
people would apprecite UTF-8-SIG detection, since this encoding is common on 
Windows.

> BTW: I haven't looked at your implementation, but what happens
> when your BOM check fails ? Will the implementation add the
> already read bytes back to a buffer ?

My implementation is done between buffer.read() and decoder.decode(data). If 
there is a BOM: set the encoding and remove the BOM bytes from the byte 
string. Otherwise, use another algorithm to choose the encoding and leave the 
byte string unchanged.

It can be seen as a codec: it works like UTF-16 and UTF-32 codecs ;-)

> AFAIK, we currently have a moratorium on changes to Python
> builtins. How does that match up with the proposed changes ?

Oh yes, I forgot the moratorium. In all solutions, some of them don't change 
the API. Eg. Antoine proposed to leave the API unchanged: open(file) => 
open(file) :-) I don't know if it's compatible with the moratorium or not.

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 16:05:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 15:05:45 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <loom.20100109T160248-501@post.gmane.org>

Walter D?rwald <walter <at> livinglogic.de> writes:
> 
> On the surface this looks like there's an encoding named "BOM", but 
> looking at your patch I found that the check is still done in 
> TextIOWrapper. IMHO the best approach would to the implement a *real* 
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
> the IO library. It could even be developed as a standalone project and 
> published in the Cheeseshop.

Sorry but this is missing the point. The point here is to improve the open()
function. I'm sure people who know about encodings are able to install the
chardet library or even whip up their own BOM detection routine...


Antoine.




From benjamin at python.org  Sat Jan  9 18:29:33 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 11:29:33 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>

On behalf of the Python development team, I'm gleeful to announce the second
alpha release of Python 2.7.

Python 2.7 is scheduled to be the last major version in the 2.x series.  It
includes many features that were first released in Python 3.1.  The faster io
module, the new nested with statement syntax, improved float repr, and the
memoryview object have been backported from 3.1. Other features include an
ordered dictionary implementation, unittests improvements, and support for ttk
Tile in Tkinter.  For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7 visit:

     http://www.python.org/download/releases/2.7/

Please note that this is a development release, intended as a preview of new
features for the community, and is thus not suitable for production use.

The 2.7 documentation can be found at:

     http://docs.python.org/2.7

Please consider trying Python 2.7 with your code and reporting any bugs you may
notice to:

     http://bugs.python.org


Have fun!

--
Benjamin Peterson
2.7 Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7's contributors)


From kmtracey at gmail.com  Sat Jan  9 19:48:12 2010
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 9 Jan 2010 13:48:12 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>

On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On behalf of the Python development team, I'm gleeful to announce the
> second
> alpha release of Python 2.7.
>
>
Well yay.  Django's test suite (1242 tests) runs with just one failure on
the 2.7 alpha 2 level, and that looks to be likely due to the improved
string/float rounding so not really a problem, just a difference.  That's
down from 104 failures and 40 errors with 2.7 alpha 1.

Note on the website page http://www.python.org/download/releases/2.7/ the
"Change log for this release" link is still pointing to the alpha 1
changelog.

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100109/d5c06f60/attachment-0004.htm>

From benjamin at python.org  Sat Jan  9 19:51:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 12:51:11 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <1afaf6161001091051kb1ba08q9b96094af16c12fa@mail.gmail.com>

2010/1/9 Karen Tracey <kmtracey at gmail.com>:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>
> Well yay.? Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.? That's
> down from 104 failures and 40 errors with 2.7 alpha 1.

Excellent!

>
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Thanks. I'll fix that.

>



-- 
Regards,
Benjamin


From skip at pobox.com  Sat Jan  9 21:00:44 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:00:44 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
Message-ID: <19272.57452.972234.643008@montanaro.dyndns.org>

How much of the Unladen Swallow cPickle speedups have been incorporated into
2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
two versions I currently care about at work - we will skip 2.5 entirely).
It appears some of their speedups may have already been merged to trunk, but
I'm not sure how much.  If a patch to merge this to 2.7 is already under
consideration I won't look at it, but I interpreted Collin Winter's response
to my query on the u-s mailing list that not everything has been done yet.

Thx,

Skip



From martin at v.loewis.de  Sat Jan  9 21:09:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 09 Jan 2010 21:09:27 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T160248-501@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
Message-ID: <4B48E277.9010408@v.loewis.de>

Antoine Pitrou wrote:
> Walter D?rwald <walter <at> livinglogic.de> writes:
>> On the surface this looks like there's an encoding named "BOM", but 
>> looking at your patch I found that the check is still done in 
>> TextIOWrapper. IMHO the best approach would to the implement a *real* 
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
>> the IO library. It could even be developed as a standalone project and 
>> published in the Cheeseshop.
> 
> Sorry but this is missing the point. The point here is to improve the open()
> function. I'm sure people who know about encodings are able to install the
> chardet library or even whip up their own BOM detection routine...

How does the requirement that it be implemented as a codec miss the
point?

FWIW, I agree with Walter that if it is provided through the encoding=
argument, it should be a codec. If it is built into the open function
(for whatever reason), it must be provided by some other parameter.

I do see the point that it becomes available to end users only when
released as part of Python. However, this *also* means that applications
won't be using it for another three years or so, since they'll have to
support older Python versions as well (unless it is integrated in the
case where no encoding is specified).

Regards,
Martin


From pjenvey at underboss.org  Sat Jan  9 21:09:29 2010
From: pjenvey at underboss.org (Philip Jenvey)
Date: Sat, 9 Jan 2010 12:09:29 -0800
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <19272.57452.972234.643008@montanaro.dyndns.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>


On Jan 9, 2010, at 12:00 PM, skip at pobox.com wrote:

> How much of the Unladen Swallow cPickle speedups have been incorporated into
> 2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
> two versions I currently care about at work - we will skip 2.5 entirely).
> It appears some of their speedups may have already been merged to trunk, but
> I'm not sure how much.  If a patch to merge this to 2.7 is already under
> consideration I won't look at it, but I interpreted Collin Winter's response
> to my query on the u-s mailing list that not everything has been done yet.

They've documented their upstream patches here:

http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

--
Philip Jenvey



From skip at pobox.com  Sat Jan  9 21:21:20 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:21:20 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
Message-ID: <19272.58688.173011.412712@montanaro.dyndns.org>


    Philip> They've documented their upstream patches here:

    Philip> http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

Thanks.  That will help immensely.

Skip



From solipsis at pitrou.net  Sat Jan  9 21:28:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:28:12 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
Message-ID: <loom.20100109T212555-508@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.




From solipsis at pitrou.net  Sat Jan  9 21:32:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:32:27 +0000 (UTC)
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <loom.20100109T213033-976@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> If a patch to merge this to 2.7 is already under
> consideration I won't look at it,

Why won't you look at it? :)
Actually, if these patches are to be merged someone should certainly look at
them, and do the (possibly) remaining work.

http://bugs.python.org/issue5683
http://bugs.python.org/issue5671

Thank you

Antoine.




From skip at pobox.com  Sat Jan  9 21:43:42 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:43:42 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
In-Reply-To: <loom.20100109T213033-976@post.gmane.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<loom.20100109T213033-976@post.gmane.org>
Message-ID: <19272.60030.25319.269597@montanaro.dyndns.org>

>>>>> "Antoine" == Antoine Pitrou <solipsis at pitrou.net> writes:

    Antoine> <skip <at> pobox.com> writes:
    >> 
    >> If a patch to merge this to 2.7 is already under
    >> consideration I won't look at it,

    Antoine> Why won't you look at it? :)

I meant I wouldn't look at developing one.

Skip


From regebro at gmail.com  Sat Jan  9 23:14:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 23:14:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>

On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Right. It seems like encoding=None is the right way to go there.
encoding='BOM' would probably only work if 'BOM' isn't an encoding but
a special tag, which is ugly.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Sun Jan 10 00:25:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 23:25:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
Message-ID: <4B49105E.303@voidspace.org.uk>

On 09/01/2010 22:14, Lennart Regebro wrote:
> On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>    
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
>>      
> Right. It seems like encoding=None is the right way to go there.
> encoding='BOM' would probably only work if 'BOM' isn't an encoding but
> a special tag, which is ugly.
>
>    
I would rather see it as the default behavior for open without an 
encoding specified.

I know Guido has expressed a preference against this so I won't continue 
to flog it.

The current behavior however is that we have a 'guessing' algorithm 
based on the platform default. Currently if you open a text file in read 
mode that has a UTF-8 signature, but the platform default is something 
other than UTF-8, then we open the file using what is likely to be the 
incorrect encoding. Looking for the signature seems to be better 
behaviour in that case.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Sun Jan 10 00:40:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Jan 2010 00:40:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <4B4913DF.7050801@v.loewis.de>

>> How does the requirement that it be implemented as a codec miss the
>> point?
> 
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Yes - however, Victor currently apparently *doesn't* want it to be the
default, but wants the user to specify encoding="BOM". If so, it isn't
the default, and it is easy to implement as a codec.

>> FWIW, I agree with Walter that if it is provided through the encoding=
>> argument, it should be a codec. If it is built into the open function
>> (for whatever reason), it must be provided by some other parameter.
> 
> Why not simply encoding=None?

I don't mind. Please re-read Walter's message - it only said that
*if* this is activated through encoding="BOM", *then* it must be
a codec, and could be on PyPI. I don't think Walter was talking about
the case "it is not activated through encoding='BOM'" *at all*.

> The default value should provide the most useful
> behaviour possible. Forcing users to choose between two different autodetection
> strategies (encoding=None and another one) is a little insane IMO.

That wouldn't disturb me much. There are a lot of things in that area
that are a little insane, starting with Microsoft Windows :-)

Regards,
Martin


From henning.vonbargen at arcor.de  Sun Jan 10 12:10:02 2010
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 10 Jan 2010 12:10:02 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <mailman.2987.1263079529.28904.python-dev@python.org>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
Message-ID: <4B49B58A.4040103@arcor.de>

If Python should support BOM when reading text files,
it should also be able to *write* such files.

An encoding="BOM" argument wouldn't help here, because
it does not specify which encoding to use actually:
UFT-8, UTF-16-LE or what?

That would be a point against encoding="BOM" and
pro an additional keyword argument "use_bom" or whatever
with the following values:

None: default (old) behaviour: don't handle BOM at all

True: reading: expect BOM (raising an exception if it's
                missing). The encoding argument must be None
                or it must match the encoding implied by the
                BOM
       writing: write a BOM. The encoding argument must be
                one of the UTF encodings.
False: reading: If a BOM is present, use it to determine the
                file encoding. The encoding argument must
                be None or it must match the encoding implied by
                the BOM. (*)
                Otherwise, use the encoding argument to determine
                the encoding.
        writing: do not write a BOM. Use the encoding argument.

(*) This is a question of taste. I think some people would prefer
     a fourth value "AUTO" instead, or to swap the behaviour of
     None and False.

Henning

P.S. To make things worse, I have sometimes seen XML files with a
UTF-8 BOM, but an XML encoding declaration of "iso-8859-1".
For such files, whatever you guess will be wrong anyway...


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From brett at python.org  Sun Jan 10 19:51:26 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 10:51:26 -0800
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
Message-ID: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>

Nick Coghlan thought I should forward this to python-dev so people are aware
of this change and why it occurred (plus it indirectly informs Guido I
finally finished the work).

---------- Forwarded message ----------
From: brett.cannon <python-checkins at python.org>
Date: Sat, Jan 9, 2010 at 18:56
Subject: [Python-checkins] r77402 - in python/trunk:
Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS Python/_warnings.c
To: python-checkins at python.org


Author: brett.cannon
Date: Sun Jan 10 03:56:19 2010
New Revision: 77402

Log:
DeprecationWarning is now silent by default.

This was originally suggested by Guido, discussed on the stdlib-sig mailing
list, and given the OK by Guido directly to me. What this change essentially
means is that Python has taken a policy of silencing warnings that are only
of interest to developers by default. This should prevent users from seeing
warnings which are triggered by an application being run against a new
interpreter before the app developer has a chance to update their code.

Closes issue #7319. Thanks to Antoine Pitrou, Ezio Melotti, and Brian Curtin
for helping with the issue.


Modified:
  python/trunk/Doc/library/warnings.rst
  python/trunk/Lib/test/test_ascii_formatd.py
  python/trunk/Lib/test/test_exceptions.py
  python/trunk/Lib/warnings.py
  python/trunk/Misc/NEWS
  python/trunk/Python/_warnings.c

Modified: python/trunk/Doc/library/warnings.rst
==============================================================================
--- python/trunk/Doc/library/warnings.rst       (original)
+++ python/trunk/Doc/library/warnings.rst       Sun Jan 10 03:56:19 2010
@@ -59,7 +59,7 @@
 | :exc:`UserWarning`               | The default category for :func:`warn`.
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`DeprecationWarning`        | Base category for warnings about
deprecated   |
-|                                  | features.
        |
+|                                  | features (ignored by default).
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`SyntaxWarning`             | Base category for warnings about
dubious      |
 |                                  | syntactic features.
        |
@@ -89,6 +89,9 @@
 standard warning categories.  A warning category must always be a subclass
of
 the :exc:`Warning` class.

+.. versionchanged:: 2.7
+   :exc:`DeprecationWarning` is ignored by default.
+

 .. _warning-filter:

@@ -148,14 +151,6 @@
 :mod:`warnings` module parses these when it is first imported (invalid
options
 are ignored, after printing a message to ``sys.stderr``).

-The warnings that are ignored by default may be enabled by passing
:option:`-Wd`
-to the interpreter. This enables default handling for all warnings,
including
-those that are normally ignored by default. This is particular useful for
-enabling ImportWarning when debugging problems importing a developed
package.
-ImportWarning can also be enabled explicitly in Python code using::
-
-   warnings.simplefilter('default', ImportWarning)
-

 .. _warning-suppress:

@@ -226,6 +221,37 @@
 entries from the warnings list before each new operation).


+Updating Code For New Versions of Python
+----------------------------------------
+
+Warnings that are only of interest to the developer are ignored by default.
As
+such you should make sure to test your code with typically ignored warnings
+made visible. You can do this from the command-line by passing
:option:`-Wd`
+to the interpreter (this is shorthand for :option:`-W default`).  This
enables
+default handling for all warnings, including those that are ignored by
default.
+To change what action is taken for encountered warnings you simply change
what
+argument is passed to :option:`-W`, e.g. :option:`-W error`. See the
+:option:`-W` flag for more details on what is possible.
+
+To programmatically do the same as :option:`-Wd`, use::
+
+  warnings.simplefilter('default')
+
+Make sure to execute this code as soon as possible. This prevents the
+registering of what warnings have been raised from unexpectedly influencing
how
+future warnings are treated.
+
+Having certain warnings ignored by default is done to prevent a user from
+seeing warnings that are only of interest to the developer. As you do not
+necessarily have control over what interpreter a user uses to run their
code,
+it is possible that a new version of Python will be released between your
+release cycles.  The new interpreter release could trigger new warnings in
your
+code that were not there in an older interpreter, e.g.
+:exc:`DeprecationWarning` for a module that you are using. While you as a
+developer want to be notified that your code is using a deprecated module,
to a
+user this information is essentially noise and provides no benefit to them.
+
+
 .. _warning-functions:

 Available Functions

Modified: python/trunk/Lib/test/test_ascii_formatd.py
==============================================================================
--- python/trunk/Lib/test/test_ascii_formatd.py (original)
+++ python/trunk/Lib/test/test_ascii_formatd.py Sun Jan 10 03:56:19 2010
@@ -4,6 +4,7 @@

 import unittest
 from test_support import check_warnings, run_unittest, cpython_only
+import warnings


 class FormatDeprecationTests(unittest.TestCase):
@@ -17,6 +18,7 @@
        buf = create_string_buffer(' ' * 100)

        with check_warnings() as w:
+            warnings.simplefilter('default')
            PyOS_ascii_formatd(byref(buf), sizeof(buf), '%+.10f',
                               c_double(10.0))
            self.assertEqual(buf.value, '+10.0000000000')

Modified: python/trunk/Lib/test/test_exceptions.py
==============================================================================
--- python/trunk/Lib/test/test_exceptions.py    (original)
+++ python/trunk/Lib/test/test_exceptions.py    Sun Jan 10 03:56:19 2010
@@ -309,6 +309,7 @@
        # BaseException.__init__ triggers a deprecation warning.
        exc = BaseException("foo")
        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('default')
            self.assertEquals(exc.message, "foo")
        self.assertEquals(len(w), 1)
        self.assertEquals(w[0].category, DeprecationWarning)

Modified: python/trunk/Lib/warnings.py
==============================================================================
--- python/trunk/Lib/warnings.py        (original)
+++ python/trunk/Lib/warnings.py        Sun Jan 10 03:56:19 2010
@@ -383,8 +383,8 @@
 # Module initialization
 _processoptions(sys.warnoptions)
 if not _warnings_defaults:
-    simplefilter("ignore", category=PendingDeprecationWarning, append=1)
-    simplefilter("ignore", category=ImportWarning, append=1)
+    for cls in (DeprecationWarning, PendingDeprecationWarning,
ImportWarning):
+        simplefilter("ignore", category=cls, append=True)
    bytes_warning = sys.flags.bytes_warning
    if bytes_warning > 1:
        bytes_action = "error"

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sun Jan 10 03:56:19 2010
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------

+- Issue #7319: Silence DeprecationWarning by default.
+
 - Issue #2335: Backport set literals syntax from Python 3.x.

 Library

Modified: python/trunk/Python/_warnings.c
==============================================================================
--- python/trunk/Python/_warnings.c     (original)
+++ python/trunk/Python/_warnings.c     Sun Jan 10 03:56:19 2010
@@ -85,10 +85,10 @@

    default_action = get_warnings_attr("defaultaction");
    if (default_action == NULL) {
-       if (PyErr_Occurred()) {
-           return NULL;
-       }
-       return _default_action;
+        if (PyErr_Occurred()) {
+            return NULL;
+        }
+        return _default_action;
    }

    Py_DECREF(_default_action);
@@ -202,12 +202,12 @@

    mod_str = PyString_AsString(filename);
    if (mod_str == NULL)
-           return NULL;
+        return NULL;
    len = PyString_Size(filename);
    if (len < 0)
        return NULL;
    if (len >= 3 &&
-       strncmp(mod_str + (len - 3), ".py", 3) == 0) {
+            strncmp(mod_str + (len - 3), ".py", 3) == 0) {
        module = PyString_FromStringAndSize(mod_str, len-3);
    }
    else {
@@ -251,7 +251,7 @@

    name = PyObject_GetAttrString(category, "__name__");
    if (name == NULL)  /* XXX Can an object lack a '__name__' attribute? */
-           return;
+        return;

    f_stderr = PySys_GetObject("stderr");
    if (f_stderr == NULL) {
@@ -341,7 +341,7 @@
        rc = already_warned(registry, key, 0);
        if (rc == -1)
            goto cleanup;
-       else if (rc == 1)
+        else if (rc == 1)
            goto return_none;
        /* Else this warning hasn't been generated before. */
    }
@@ -492,9 +492,9 @@
    /* Setup filename. */
    *filename = PyDict_GetItemString(globals, "__file__");
    if (*filename != NULL) {
-           Py_ssize_t len = PyString_Size(*filename);
+            Py_ssize_t len = PyString_Size(*filename);
        const char *file_str = PyString_AsString(*filename);
-           if (file_str == NULL || (len < 0 && PyErr_Occurred()))
+            if (file_str == NULL || (len < 0 && PyErr_Occurred()))
            goto handle_error;

        /* if filename.lower().endswith((".pyc", ".pyo")): */
@@ -506,10 +506,10 @@
                tolower(file_str[len-1]) == 'o'))
        {
            *filename = PyString_FromStringAndSize(file_str, len-1);
-               if (*filename == NULL)
-                       goto handle_error;
-           }
-           else
+            if (*filename == NULL)
+                goto handle_error;
+        }
+        else
            Py_INCREF(*filename);
    }
    else {
@@ -536,8 +536,8 @@
            else {
                /* embedded interpreters don't have sys.argv, see bug
#839151 */
                *filename = PyString_FromString("__main__");
-                   if (*filename == NULL)
-                       goto handle_error;
+                if (*filename == NULL)
+                    goto handle_error;
            }
        }
        if (*filename == NULL) {
@@ -839,26 +839,29 @@
 static PyObject *
 init_filters(void)
 {
-    PyObject *filters = PyList_New(3);
+    PyObject *filters = PyList_New(4);
    const char *bytes_action;
    if (filters == NULL)
        return NULL;

    PyList_SET_ITEM(filters, 0,
+                    create_filter(PyExc_DeprecationWarning, "ignore"));
+    PyList_SET_ITEM(filters, 1,
                    create_filter(PyExc_PendingDeprecationWarning,
"ignore"));
-    PyList_SET_ITEM(filters, 1, create_filter(PyExc_ImportWarning,
"ignore"));
+    PyList_SET_ITEM(filters, 2, create_filter(PyExc_ImportWarning,
"ignore"));
    if (Py_BytesWarningFlag > 1)
        bytes_action = "error";
    else if (Py_BytesWarningFlag)
        bytes_action = "default";
    else
        bytes_action = "ignore";
-    PyList_SET_ITEM(filters, 2, create_filter(PyExc_BytesWarning,
+    PyList_SET_ITEM(filters, 3, create_filter(PyExc_BytesWarning,
                    bytes_action));

    if (PyList_GET_ITEM(filters, 0) == NULL ||
        PyList_GET_ITEM(filters, 1) == NULL ||
-        PyList_GET_ITEM(filters, 2) == NULL) {
+        PyList_GET_ITEM(filters, 2) == NULL ||
+        PyList_GET_ITEM(filters, 3) == NULL) {
        Py_DECREF(filters);
        return NULL;
     }
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db6da5fd/attachment-0004.htm>

From victor.stinner at haypocalc.com  Sun Jan 10 20:22:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 10 Jan 2010 20:22:10 +0100
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
Message-ID: <201001102022.10259.victor.stinner@haypocalc.com>

Hi,

Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
> Nick Coghlan thought I should forward this to python-dev so people are
>  aware of this change and why it occurred (plus it indirectly informs Guido
>  I finally finished the work).

Thanks to copy this mail to the python-dev mailing list.

What is the recommanded way to get the previous behaviour (print deprecation 
warnings once)? Is there a command line option? Is it "-W default"?

-- 
Victor Stinner
http://www.haypocalc.com/


From benjamin at python.org  Sun Jan 10 20:23:54 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 13:23:54 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <201001102022.10259.victor.stinner@haypocalc.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
	<201001102022.10259.victor.stinner@haypocalc.com>
Message-ID: <1afaf6161001101123g366d80dbjeaa80f1a632605e2@mail.gmail.com>

2010/1/10 Victor Stinner <victor.stinner at haypocalc.com>:
> Hi,
>
> Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
>> Nick Coghlan thought I should forward this to python-dev so people are
>> ?aware of this change and why it occurred (plus it indirectly informs Guido
>> ?I finally finished the work).
>
> Thanks to copy this mail to the python-dev mailing list.
>
> What is the recommanded way to get the previous behaviour (print deprecation
> warnings once)? Is there a command line option? Is it "-W default"?

That commit conveniently adds documentation answering that question. :)



-- 
Regards,
Benjamin


From nas at arctrix.com  Sun Jan 10 20:30:09 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:30:09 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <hid9s1$i05$1@ger.gmane.org>

Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm gleeful to announce
> the second alpha release of Python 2.7.

Thanks to everyone who contributed.

> Python 2.7 is scheduled to be the last major version in the 2.x
> series.

Has this really been decided already? Maybe I missed it.  In my
opinion, it does Python's reputation harm to make such a statement.
Conservative users (probably the vast majority of Python users)
don't like to hear that software they are considering using is
nearing the end of its life.  What does it gain us to announce that
the 2.x branch is dead aside from bugfixes?

I propose that the 2.x branch be treated like 2.x.y branches: as
long as there is sufficient volunteer labour, it should continue to
live.  In order to avoid wasted development effort, it would be
prudent to announce that unless a 2.8 release manager steps up,
whatever is committed to the 2.x branch after the 2.7 release may
never get released.

Said another way, it's okay for the Python developers to decide to
abandon 2.x and put their efforts into 3.x. It's not okay for them
to prevent others from continuing to work on 2.x or to somehow make
2.x look worse so 3.x looks better. Python 3 needs to stand on its
own terms and I'm confident it can.

Best regards,

  Neil



From brett at python.org  Sun Jan 10 21:09:08 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:09:08 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>

On Sun, Jan 10, 2010 at 11:30, Neil Schemenauer <nas at arctrix.com> wrote:

> Benjamin Peterson <benjamin at python.org> wrote:
> > On behalf of the Python development team, I'm gleeful to announce
> > the second alpha release of Python 2.7.
>
> Thanks to everyone who contributed.
>
> > Python 2.7 is scheduled to be the last major version in the 2.x
> > series.
>
> Has this really been decided already? Maybe I missed it.


More or less. It was first discussed at the language summit last year and
has come up here a couple of times. If needed we can make it official in
terms of lifetime of 2.7, etc. at the language summit this year.


>  In my
> opinion, it does Python's reputation harm to make such a statement.
> Conservative users (probably the vast majority of Python users)
> don't like to hear that software they are considering using is
> nearing the end of its life.  What does it gain us to announce that
> the 2.x branch is dead aside from bugfixes?
>
> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.
>
> Said another way, it's okay for the Python developers to decide to
> abandon 2.x and put their efforts into 3.x. It's not okay for them
> to prevent others from continuing to work on 2.x or to somehow make
> 2.x look worse so 3.x looks better. Python 3 needs to stand on its
> own terms and I'm confident it can.
>

I don't think ending the 2.x series at 2.7 makes it look bad compared to
3.2; it's simply the end of a development line like any other software
project. I suspect 2.7 will have a protracted bugfix window because so much
code runs on 2.x exclusively at the moment. And if core developers want to
continue to backport fixes past two years  from release they can (or however
long we decide to officially support 2.7).

No one is saying people still can't work on the code, just that python-dev
as an entity is not going to focus its effort on the 2.x series anymore and
people should not rely upon us to continue to focus new development efforts
in that series. If there are core developers who want to continue to do
bugfix releases then that's fine and I am happy to flag patches as needing
backports and let other's do the work after the standard two year
maintenance cycle, but I know I do not want to be held accountable as a core
developer to keep the 2.x going indefinitely. Maintaining four branches is
enough of a reason in my book to not keep the 2.x series going.

If there really is an outcry on this we can re-visit the issue, but as of
right now we need to move forward at some point and 2.7 seems like that good
point.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/4bff0434/attachment-0004.htm>

From ncoghlan at gmail.com  Sun Jan 10 21:23:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 11 Jan 2010 06:23:27 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A373F.9050601@gmail.com>

Neil Schemenauer wrote:
> In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

The announcement is precisely to avoid the situation where people commit
new features to the 2.x main line of development (after the 2.7
maintenance branch is created) in the expectation that they will be
released as part of a hypothetical 2.8 release.

Whether that info needs to be in each and every 2.7 announcement...
probably not. It isn't really info for users of Python, just for
developers of Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Sun Jan 10 21:25:29 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:25:29 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
Message-ID: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>

Michael has given me the hg transition/stdlib time slot at the language
summit this year. In regards to that I plan to lead a discussion on:

* where we are at w/ the Hg transition (Dirkjan should be there and I did a
blog post on this topic recently:
http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
* argparse (PEP 389)
* brief mention on still wanting to break out the stdlib from CPython
* an official policy on extension modules? (i.e. must have a pure Python
implementation which can mean a ctypes implementation unless given an
explicit waiver)

That's everything from a stdlib perspective. I have half-baked ideas, but
nothing concrete enough to bring up unless people really want to discuss
stuff like how to potentially standardize module deprecation warnings, etc.

In terms of me personally, I do plan to bring up at some point during the
summit these points which don't squarely fit during my time slot:

* an official unofficial policy on how new proposed features should come to
us (i.e. working code to python-ideas with a shell of a PEP that includes
abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
* any changes needed to the issue tracker to help with the workflow? (stage
field seems like a failed experiment and we now have several effective
triage people who can help w/ guiding changes)

If there is something missing from the stdlib discussion that you think
should be brought up at the summit let me know. And if there is something
here you want to discuss before the summit let me know and I can start a
separate thread on it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/f16584e2/attachment-0004.htm>

From dirkjan at ochtman.nl  Sun Jan 10 22:52:00 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 10 Jan 2010 22:52:00 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:25, Brett Cannon <brett at python.org> wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)

Unfortunately my flight doesn't land until about the time the
Mercurial slot ends, so while I'll be there later on that afternoon
for sprinting or questions or anything, I won't be at the actual
Mercurial talk at the summit.

Cheers,

Dirkjan


From fuzzyman at voidspace.org.uk  Sun Jan 10 23:27:58 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 10 Jan 2010 22:27:58 +0000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
Message-ID: <4B4A546E.8010808@voidspace.org.uk>

On 10/01/2010 21:52, Dirkjan Ochtman wrote:
> On Sun, Jan 10, 2010 at 21:25, Brett Cannon<brett at python.org>  wrote:
>    
>> Michael has given me the hg transition/stdlib time slot at the language
>> summit this year. In regards to that I plan to lead a discussion on:
>> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
>> blog post on this topic recently:
>> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
>>      
> Unfortunately my flight doesn't land until about the time the
> Mercurial slot ends, so while I'll be there later on that afternoon
> for sprinting or questions or anything, I won't be at the actual
> Mercurial talk at the summit.
>
>    
We will probably be in 'open discussion' when you arrive so it will 
still be good to hear from you on the topic and there maybe questions 
that can't be answered until you arrive... :-)

Michael

> Cheers,
>
> Dirkjan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Mon Jan 11 00:02:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:02:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A5C69.7090007@v.loewis.de>

> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

I think it's more difficult than that. It also takes developer effort
to *commit* to the trunk. So if the trunk continued to live, would
it still be ok if I closed a 2.x feature request as "won't fix", or
only committed the new feature to the 3.x branch?

As for old branches: they *don't* live in the way you claim (i.e.
remain open with changes potentially just not released). Instead,
at some point, they are frozen to bug-fix only, then to security-fix
only, and then to no change at all.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 00:07:58 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:07:58 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A373F.9050601@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>
Message-ID: <4B4A5DCE.3070909@v.loewis.de>

> The announcement is precisely to avoid the situation where people commit
> new features to the 2.x main line of development (after the 2.7
> maintenance branch is created) in the expectation that they will be
> released as part of a hypothetical 2.8 release.
> 
> Whether that info needs to be in each and every 2.7 announcement...
> probably not. It isn't really info for users of Python, just for
> developers of Python.

I think the announcement is indeed also for the users, and I would
prefer it to stay in the release announcements, so that people will
know for sure (and speak up) before the 2.7 release happens.

As for decisions: I don't think there was an official BDFL pronouncent,
but I recall Guido posting a message close to that, proposing that 2.7
will be a release that will see bug fix releases for an indefinite
period of time (where indefinite != infinite). This was shortly after
him proposing that perhaps we shouldn't make a 2.7 release at all, and
stop at 2.6.

As for such a decision giving a bad light on Python: I don't think that
will be the case. Instead, I hear many users surprised for how long
we have been maintaining to parallel versions - "that must have taken
a lot of effort". So everybody will likely understand that enough is
enough.

Regards,
Martin


From benjamin at python.org  Mon Jan 11 01:07:35 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 18:07:35 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>

Consider this program:

class Descr(object):
    def __init__(self, name):
        self.name = name
    def __set__(self, instance, what):
        instance.__dict__[self.name] = what

class X(object):
    attr = Descr("attr")

x = X()
print(x.attr)
x.attr = 42
print(x.attr)

It gives in output:

<__main__.Descr object at 0x7fe1c9b28150>
42

The documentation [1] says that Descr is a data descriptor because it
defines the __set__ method. It also states that data descriptors
always override the value in the instance dictionary. So, the second
line should also be the descriptor object according to the
documentation.

My question is: Is this a doc bug or a implementation bug? If the
former, it will be the description of a data descriptor much less
consistent, since it will require that a __get__ method be present,
too. If the latter, the fix may break some programs relying on the
ability to "cache" a value in the instance dictionary.

[1] http://docs.python.org/reference/datamodel#invoking-descriptors


-- 
Regards,
Benjamin


From amauryfa at gmail.com  Mon Jan 11 01:51:09 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 11 Jan 2010 01:51:09 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>

Hi,

2010/1/11 Benjamin Peterson <benjamin at python.org>:
> Consider this program:
>
> class Descr(object):
> ? ?def __init__(self, name):
> ? ? ? ?self.name = name
> ? ?def __set__(self, instance, what):
> ? ? ? ?instance.__dict__[self.name] = what
>
> class X(object):
> ? ?attr = Descr("attr")
>
> x = X()
> print(x.attr)
> x.attr = 42
> print(x.attr)
>
> It gives in output:
>
> <__main__.Descr object at 0x7fe1c9b28150>
> 42
>
> The documentation [1] says that Descr is a data descriptor because it
> defines the __set__ method. It also states that data descriptors
> always override the value in the instance dictionary. So, the second
> line should also be the descriptor object according to the
> documentation.
>
> My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
>
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

Quoting the documentation:
"""Normally, data descriptors define both __get__() and __set__(),
while non-data descriptors have just the __get__() method.
"""
Your example is neither a data descriptor nor a non-data descriptor...

The thing that worries me a bit is the "x.attr" returning the Descr
object. Descriptors should remain at the class level, and instance
should only see values. I'd prefer an AttributeError in this case.

-- 
Amaury Forgeot d'Arc


From benjamin at python.org  Mon Jan 11 02:00:32 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 19:00:32 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
Message-ID: <1afaf6161001101700u21006708l2ef62bfa257e4ce7@mail.gmail.com>

2010/1/10 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...

See the footnote: http://docs.python.org/reference/datamodel#id7

>
> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.

Far too late for that, I'm afraid.



-- 
Regards,
Benjamin


From nas at arctrix.com  Mon Jan 11 02:44:57 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:44:57 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
Message-ID: <20100111014457.GA5289@arctrix.com>

On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> I don't think ending the 2.x series at 2.7 makes it look bad
> compared to 3.2; it's simply the end of a development line like
> any other software project. I suspect 2.7 will have a protracted
> bugfix window because so much code runs on 2.x exclusively at the
> moment.

I would guess over 99% of all Python code written doesn't run on
Python 3.  Given that, I think it is premature to close the door on
new major versions of Python 2.x. Also, we as a project should be
careful not to present the image that Python 2.x will not be
supported in the future.

> If there really is an outcry on this we can re-visit the issue,
> but as of right now we need to move forward at some point and 2.7
> seems like that good point.

I think that's bad PR.  If I had a successful product, I would not
announce its end of life just to see how many customers scream and
then decide if I should devote more resources to continue
maintaining it.

IMHO, the release notes should say something like:

     After the Python 2.7 release, the focus of Python development
     will be on Python 3.  There will continue to be maintainance
     releases of Python 2.x.


trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil


From tjreedy at udel.edu  Mon Jan 11 03:26:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Jan 2010 21:26:34 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <hie28p$joo$1@ger.gmane.org>

On 1/10/2010 8:44 PM, Neil Schemenauer wrote:
> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
>> I don't think ending the 2.x series at 2.7 makes it look bad
>> compared to 3.2; it's simply the end of a development line like
>> any other software project. I suspect 2.7 will have a protracted
>> bugfix window because so much code runs on 2.x exclusively at the
>> moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.

If the removal of old features had been done in the 2.x series, as once 
planned (Guido originally proposed removing the old meaning of int / int 
in 2.5) the same more or less would be true of 2.7. It is past time for 
other old and now duplicated features to be removed also.

   Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>
>> If there really is an outcry on this we can re-visit the issue,
>> but as of right now we need to move forward at some point and 2.7
>> seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.

Python is not being ended, but upgraded (with bloating duplications 
removed). Think of 3.1 as 2.8 with a new name.

tjr



From lrekucki at gmail.com  Mon Jan 11 04:26:48 2010
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Mon, 11 Jan 2010 04:26:48 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <dfbefb091001101926l366043f8mb95f196ba14f9c9@mail.gmail.com>

> Date: Mon, 11 Jan 2010 01:51:09 +0100
> From: "Amaury Forgeot d'Arc" <amauryfa at gmail.com>
> To: Benjamin Peterson <benjamin at python.org>
> Cc: Python Dev <python-dev at python.org>
> Subject: Re: [Python-Dev] Data descriptor doc/implementation
> ? ? ? ?inconsistency
> Message-ID:
> ? ? ? ?<e27efe131001101651y68e1da25je2a8d02f5c62ef19 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> 2010/1/11 Benjamin Peterson <benjamin at python.org>:
>> Consider this program:
>>
>> class Descr(object):
>> ? ?def __init__(self, name):
>> ? ? ? ?self.name = name
>> ? ?def __set__(self, instance, what):
>> ? ? ? ?instance.__dict__[self.name] = what
>>
>> class X(object):
>> ? ?attr = Descr("attr")
>>
>> x = X()
>> print(x.attr)
>> x.attr = 42
>> print(x.attr)
>>
>> It gives in output:
>>
>> <__main__.Descr object at 0x7fe1c9b28150>
>> 42
>>
>> The documentation [1] says that Descr is a data descriptor because it
>> defines the __set__ method. It also states that data descriptors
>> always override the value in the instance dictionary. So, the second
>> line should also be the descriptor object according to the
>> documentation.
>>
>> My question is: Is this a doc bug or a implementation bug? If the
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...
Actually, there is this footnote [1]:

""" A descriptor can define any combination of __get__(), __set__()
and __delete__(). If it does not define __get__(), then accessing the
attribute even on an instance will return the descriptor object
itself. If the descriptor defines __set__() and/or __delete__(), it is
a data descriptor; if it defines neither, it is a non-data descriptor.
"""

Which would mean Descr is actually a data descriptor without a
__get__(), so x.attr should always return the descriptor object itself
(at least in the docs).

> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.
>
> --
> Amaury Forgeot d'Arc

[1] http://docs.python.org/reference/datamodel#id7
-- 
Lukasz Rekucki


From brett at python.org  Mon Jan 11 04:46:04 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 19:46:04 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
Message-ID: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>

On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> > I don't think ending the 2.x series at 2.7 makes it look bad
> > compared to 3.2; it's simply the end of a development line like
> > any other software project. I suspect 2.7 will have a protracted
> > bugfix window because so much code runs on 2.x exclusively at the
> > moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x.


Well yeah; Python 3.0 is just over a year old with 3.1 -- the first robust
3.x release - is about six months old. From the beginning uptake was
expected to take years, not months, so saying that 3.x is not popular enough
seems premature.


> Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>

No one has said bugfixes will cease.


>
> > If there really is an outcry on this we can re-visit the issue,
> > but as of right now we need to move forward at some point and 2.7
> > seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.
>
>
I never said that I wanted to make this announcement specifically to provoke
outcries. I said if there happened to be outcries we could possibly visit
the issue again. Basically I was being considerate and trying to leave the
door open to discuss things in the future even though I don't see the
situation changing.


> IMHO, the release notes should say something like:
>
>     After the Python 2.7 release, the focus of Python development
>     will be on Python 3.  There will continue to be maintainance
>     releases of Python 2.x.
>

No because that suggests new features will be coming to 2.x which is not
going to happen. If you want to say there will be continual bugfix releases
for 2.7 as is par the course for Python and that the number of bugfix
releases might be more than normal then I am okay with that.


>
>
> trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil
>

That came and went already a couple months ago when we discussed stopping at
2.6 instead of 2.7 (see the various threads at
http://mail.python.org/pipermail/python-dev/2009-November/thread.html).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db586677/attachment-0004.htm>

From nas at arctrix.com  Mon Jan 11 05:05:07 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:05:07 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
Message-ID: <20100111040507.GA7200@arctrix.com>

On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> >     After the Python 2.7 release, the focus of Python development
> >     will be on Python 3.  There will continue to be maintainance
> >     releases of Python 2.x.
> 
> No because that suggests new features will be coming to 2.x which is not
> going to happen. If you want to say there will be continual bugfix releases
> for 2.7 as is par the course for Python and that the number of bugfix
> releases might be more than normal then I am okay with that.

Are you are saying that if someone steps up to merge the Unladen
Swallow features into a 2.8 release and someone also steps up to cut
the release that they will be prevented from doing so?

Also, are you presuming to channel the BDFL or was this dicussed
somewhere other than python-dev? Maybe I'm overreacting but I get
the feeling that the larger and less active segment of the Python
develpment team hasn't been involved in these decisions.

Best regards,

  Neil


From nas at arctrix.com  Mon Jan 11 05:17:54 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:17:54 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5C69.7090007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
Message-ID: <20100111041754.GB7200@arctrix.com>

On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
> [...] would it still be ok if I closed a 2.x feature request as
> "won't fix", or only committed the new feature to the 3.x branch?

Yes.  Non-bugfix development on 2.x would optional (i.e. done by
people who want to spend the time). Since language changes are now
out (an initiative I completely support), the majority of non-bugfix
changes would be optimizations and platform porting.

Regards,

  Neil


From brett at python.org  Mon Jan 11 06:06:15 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 21:06:15 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111040507.GA7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
Message-ID: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>

On Sun, Jan 10, 2010 at 20:05, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> > On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> > >     After the Python 2.7 release, the focus of Python development
> > >     will be on Python 3.  There will continue to be maintainance
> > >     releases of Python 2.x.
> >
> > No because that suggests new features will be coming to 2.x which is not
> > going to happen. If you want to say there will be continual bugfix
> releases
> > for 2.7 as is par the course for Python and that the number of bugfix
> > releases might be more than normal then I am okay with that.
>
> Are you are saying that if someone steps up to merge the Unladen
> Swallow features into a 2.8 release and someone also steps up to cut
> the release that they will be prevented from doing so?
>
>
I honestly don't know, but it's a possibility just like any other new
feature request. If people start taking the carrots we have added to 3.x and
backporting them to keep the 2.x series alive you are essentially making the
3.x DOA by negating its benefits which I personally don't agree with.


> Also, are you presuming to channel the BDFL or was this dicussed
> somewhere other than python-dev?


This was discussed; see the November 2009 threads. Guido actually suggested
ending the 2.x branch at 2.6 until people spoke up about the amount of stuff
already backported to 2.7 from 3.x because it was being assumed to be the
end of the series to warrant keeping it to help transition to 2.7.


> Maybe I'm overreacting but I get
> the feeling that the larger and less active segment of the Python
> develpment team hasn't been involved in these decisions.
>

This all came up in November from the 3rd through the 6th (four days) over a
ton of email traffic. This was not a snap decision but a heated discussion
where even Guido participated that culminated with the decision to stop at
2.7 for the 2.x series; this was not a smoked-filled room decision. I'm
sorry if people missed it when they weren't looking, but python-dev is
supposed to be the place to watch for this kind of stuff. I don't know how
else to bring this stuff to people's attention short of also on
python-committers, but developers are basically expected to be on both lists
so I don't see where anyone did anything wrong in terms of informing
developers.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/18c3bcd7/attachment-0004.htm>

From nas at arctrix.com  Mon Jan 11 06:27:13 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 23:27:13 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
Message-ID: <20100111052713.GA8211@arctrix.com>

On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> If people start taking the carrots we have added to 3.x and
> backporting them to keep the 2.x series alive you are essentially
> making the 3.x DOA by negating its benefits which I personally
> don't agree with.

I think we have got to the heart of our disagreement. Assume that
some superhuman takes all the backwards compatible goodies from 3.x
and merges them into 2.x. If that really would kill off Python 3
then I would proclaim it a project that deserved to die. Why should
the backwards incompatible changes be endured if they cannot justify
their existance by the benefit they provide?

I guess I have more confidence in Python 3 than you do. I don't see
why Python 2.x needs to be artificially limited so that Python 3 can
benefit.

Regards,

  Neil


From brett at python.org  Mon Jan 11 07:30:49 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 22:30:49 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
Message-ID: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:27, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> > If people start taking the carrots we have added to 3.x and
> > backporting them to keep the 2.x series alive you are essentially
> > making the 3.x DOA by negating its benefits which I personally
> > don't agree with.
>
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x. If that really would kill off Python 3
> then I would proclaim it a project that deserved to die. Why should
> the backwards incompatible changes be endured if they cannot justify
> their existance by the benefit they provide?
>
>
When I said "carrots" I meant everything that makes Python 3 the best
version of Python, including the backward-incompatible changes.

I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.
>

I don't view it as artificial but simply a focusing of the development team
on what has been determined as the future of Python by Guido and letting go
of the past.

IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev
saying "Python 3 is the future, but we are keeping the old Python 2.x around
because we don't have *that* much faith in the future we have laid out".
That's poison to Python 3 by showing a lack of confidence in the direction
that the BDFL and python-dev as a group has chosen. Now I could be wrong and
there could actually be a large number of active contributors who want to
keep the 2.x series going, but based on the discussion that occurred the
last time this came up I believe the guys who are keeping Python running are
ready to move on.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/36448e5f/attachment-0004.htm>

From regebro at gmail.com  Mon Jan 11 08:08:14 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 08:08:14 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <319e029f1001102308p5de87cber2131f3aec1abc9f0@mail.gmail.com>

On Mon, Jan 11, 2010 at 06:27, Neil Schemenauer <nas at arctrix.com> wrote:
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x.

The superhumans of core developers pretty much already have. That's
pretty much 2.7 you are talking about. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From chrism at plope.com  Mon Jan 11 08:27:03 2010
From: chrism at plope.com (Chris McDonough)
Date: Mon, 11 Jan 2010 02:27:03 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
Message-ID: <4B4AD2C7.1050703@plope.com>

Brett Cannon wrote:
> IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev 
> saying "Python 3 is the future, but we are keeping the old Python 2.x 
> around because we don't have *that* much faith in the future we have 
> laid out". That's poison to Python 3 by showing a lack of confidence in 
> the direction that the BDFL and python-dev as a group has chosen. Now I 
> could be wrong and there could actually be a large number of active 
> contributors who want to keep the 2.x series going, but based on the 
> discussion that occurred the last time this came up I believe the guys 
> who are keeping Python running are ready to move on.

I think this is mostly just a branding issue.  Once the folks who have 
historically kept Python 2 running really *do* move on, there will be other 
folks who will step up and become maintainers out of necessity: those stuck on 
old platforms, permanent stragglers, people who for whatever reason actually 
like Python 2 better, etc.  It's just going to happen, there's really nothing 
anybody can do about it.

I don't think there's anything wrong with this.  If such a group of folks wants 
to get together and create another Python 2.X release, there's nothing stopping 
them from doing so except the above branding declaration.  I'd personally not 
close the door on allowing these folks to call such an effort "Python 2".

- C



From martin at v.loewis.de  Mon Jan 11 09:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:06:16 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
	<20100111041754.GB7200@arctrix.com>
Message-ID: <4B4ADBF8.3030803@v.loewis.de>

Neil Schemenauer wrote:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time). 

I think *that* would give a very bad impression of Python. Depending
whom you deal with, the new feature you want may or may not get
added to the code base. Contributors would feel even more stranded
than they do now, where it may take several years to get a patch
reviewed, as you then could submit a patch, and pray that a comitter
reviews it who believes in future 2.x releases.

The point of setting policies is that it gives every user (contributors,
committers, and "end-user" developers) a reliable foundation for
expectations.

Regards,
Martin


From arcriley at gmail.com  Mon Jan 11 09:06:10 2010
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 11 Jan 2010 03:06:10 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AD2C7.1050703@plope.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com> 
	<4B4AD2C7.1050703@plope.com>
Message-ID: <bad82a81001110006n3cacd953g98fb25e96330ee89@mail.gmail.com>

after all these years, some people still run AmigaOS.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/4fdcbb4e/attachment-0004.htm>

From martin at v.loewis.de  Mon Jan 11 09:11:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:11:25 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <4B4ADD2D.6070906@v.loewis.de>

> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.

Why that? It is a fact: 2.x will not be supported, in some future.
Should we lie to users?

>      After the Python 2.7 release, the focus of Python development
>      will be on Python 3.  There will continue to be maintainance
>      releases of Python 2.x.

It shouldn't say that, because it wouldn't be true.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 09:18:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:18:30 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <4B4ADED6.5080207@v.loewis.de>

> I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.

Because it takes too much time. Too much of my time, but apparently
also too much of other people's time.

Of course, the less active fraction of Python contributors may not
notice, since they just chose to not contribute (which, of course,
is fine). However, asking me to work twice as much as I want to
on the project to keep two branches alive is just unfair.

This has nothing to do with pushing 3.x, but all with managing
available manpower and still providing quality software.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 09:42:51 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 09:42:51 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>

This is what it says now:

"Python 2.7 is scheduled to be the last major version in the 2.x
series before it moves into 5 years of bugfix-only mode. "

And during this discussion it has been noted that others than the core
python team might pick up Python 2 and make releases. So maybe we can
and this discussion by changing that line in future releases to be:

"Python 2.7 is scheduled to be the last major version in the 2.x
series released by the Python Software Foundation before it moves into
5 years of bugfix-only mode. "

That doesn't exclude *others* making a Python 2.8 that could include
all sorts of crazy features. :)

This is mainly just to get the pointless discussion over with. The
current Python core team don't want to make a 2.8, so that will not
happen. Someone else might, but as Chris points out, they won't step
up until they have to, which is probably at least two years from now.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 10:06:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 10:06:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
Message-ID: <4B4AEA25.7010100@v.loewis.de>

> "Python 2.7 is scheduled to be the last major version in the 2.x
> series released by the Python Software Foundation before it moves into
> 5 years of bugfix-only mode. "
> 
> That doesn't exclude *others* making a Python 2.8 that could include
> all sorts of crazy features. :)
> 
> This is mainly just to get the pointless discussion over with.

I know you are just looking for a compromise, but this shouldn't be it:
the PSF has deliberately stayed out of the actual Python engineering,
so the release that Benjamin makes is not done by the PSF (but by
Benjamin and his contributors :-).

I like the wording as it is (although I find the promise of five years
of bug fix releases a bit too strong).

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 10:19:24 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 10:19:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AEA25.7010100@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
Message-ID: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>

On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I know you are just looking for a compromise, but this shouldn't be it:
> the PSF has deliberately stayed out of the actual Python engineering,
> so the release that Benjamin makes is not done by the PSF (but by
> Benjamin and his contributors :-).

Hm. Yeah. That's right of course. I started with saying "official",
but then I thought "official according to who?". :) So I changed it to
mentioning PSF, but that doesn't work either. I guess the current
writing as as good as it gets, unless we change "scheduled" to
"expected" or something.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Mon Jan 11 10:42:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 Jan 2010 10:42:05 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A5C69.7090007@v.loewis.de> <20100111041754.GB7200@arctrix.com>
Message-ID: <hierpd$dm1$1@ger.gmane.org>

Neil Schemenauer, 11.01.2010 05:17:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time).

Note that there's also the time it takes to make a release (usually 
involving at least one beta release), plus the possibility of introducing 
bugs while adding new features or even while fixing agreed bugs. All of 
this takes time in addition to the time people might want to invest in 
'real' development on the 2.x trunk.

Stefan



From stephen at xemacs.org  Mon Jan 11 10:59:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 11 Jan 2010 18:59:57 +0900
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <8763793qsi.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Schemenauer writes:
 > On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:

 > > If people start taking the carrots we have added to 3.x and
 > > backporting them to keep the 2.x series alive you are essentially
 > > making the 3.x DOA by negating its benefits which I personally
 > > don't agree with.

Well, I think it's *worse* than that, and I don't think you really
mean "DOA", anyway.  (Feel free to correct me, of course.)

The problem I see with backporting lots of stuff, and/or adding new
features that aren't in 3.0, to 2.x is that it will make 2.x even
cruftier, when it was already crufty enough that Guido (and almost all
of python-dev) bit the bullet and said "backward compatibility is no
excuse for keeping something in 3.0".

That surely means that a lot of python-dev denizens will declare
non-support 2.x for x > 7.  It's not going to be the gradual migration
we've seen over the past few months as active people start to spend
more and more time on 3 vs. 2; it will be a watershed.  Especially if
these are new features merged from outside that the "small active
segment" doesn't know anything about.  From the users' point of view,
that amounts to a *fork*, even if it's internal and "friendly".

 > I think we have got to the heart of our disagreement. Assume that
 > some superhuman takes all the backwards compatible goodies from 3.x
 > and merges them into 2.x.

Isn't that a bit ridiculous?  I just don't see any evidence that
anything like that is going to happen.  Worse, if we *assume* it will
happen, I don't see any way to assess whether (1) Python 3 goes belly
up, (2) there's an effective fork confusing the users and draining the
energy of python-dev, or (3) everybody goes "wow!" because it's so
cool that everybody wants to keep maintaining an extra 3 branches
indefinitely.

My opinion is that given the clear direction the "small active
segment" is going, telling the users anything but what Brett proposed
is disinformation.

 > I guess I have more confidence in Python 3 than you do. I don't see
 > why Python 2.x needs to be artificially limited so that Python 3 can
 > benefit.

It's not for Python 3, which you, I, and I'm pretty sure Brett-in-his-
heart-of-hearts agree can take care of itself because it is *better*
than Python 2.  It's for Python, and for the Python community.


From walter at livinglogic.de  Mon Jan 11 11:37:56 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 11:37:56 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001091438.43576.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
Message-ID: <4B4AFF84.7070206@livinglogic.de>

On 09.01.10 14:38, Victor Stinner wrote:

> Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
>>> Good idea, I choosed open(filename, encoding="BOM").
>>
>> On the surface this looks like there's an encoding named "BOM", but
>> looking at your patch I found that the check is still done in
>> TextIOWrapper. IMHO the best approach would to the implement a *real*
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
>> the IO library. It could even be developed as a standalone project and
>> published in the Cheeseshop.
> 
> Why not, this is another solution to the point (2) (Check for a BOM while 
> reading or detect it before?). Which encoding would be used if there is not 
> BOM? UTF-8 sounds like a good choice.

UTF-8 might be a good choice, are the failback could be specified in the
encoding name, i.e.

   open("file.txt", encoding="BOM-UTF-8")

falls back to UTF-8, if there's no BOM at the start.

This could be implemented via a custom codec search function (see
http://docs.python.org/library/codecs.html#codecs.register for more info).

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 12:12:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:12:20 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4AFF84.7070206@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
Message-ID: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>

On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
> UTF-8 might be a good choice

No, fallback if there is no BOM should be the local settings, just as
fallback is today if you don't specify a codec.
I mean, what if you want to look for a BOM but fall back to something
else? How far will we go with encoding special information in the
codecs names? codec='BOM else UTF-16 else locale'? :-)

BOM is not a locale, and should not be a locale. Having a locale
called BOM is wrong per se. It should either be default to look for a
BOM when codec=None, or a special parameter. If none of these are
desired, then we need a special function that takes a filename or file
handle, and looks for a BOM and returns the codec found or None. But
I find that much less natural and obvious than checking the BOM when codec=None.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From regebro at gmail.com  Mon Jan 11 12:13:00 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:13:00 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <319e029f1001110313u1d839deodfe06a6c7ec423cf@mail.gmail.com>

On Mon, Jan 11, 2010 at 12:12, Lennart Regebro <regebro at gmail.com> wrote:
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se.

D'oh! I mean codec here obviously.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 13:29:04 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 13:29:04 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4913DF.7050801@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>
Message-ID: <4B4B1990.90705@livinglogic.de>

On 10.01.10 00:40, "Martin v. L?wis" wrote:
>>> How does the requirement that it be implemented as a codec miss the
>>> point?
>>
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
> 
> Yes - however, Victor currently apparently *doesn't* want it to be the
> default, but wants the user to specify encoding="BOM". If so, it isn't
> the default, and it is easy to implement as a codec.
> 
>>> FWIW, I agree with Walter that if it is provided through the encoding=
>>> argument, it should be a codec. If it is built into the open function
>>> (for whatever reason), it must be provided by some other parameter.
>>
>> Why not simply encoding=None?
> 
> I don't mind. Please re-read Walter's message - it only said that
> *if* this is activated through encoding="BOM", *then* it must be
> a codec, and could be on PyPI. I don't think Walter was talking about
> the case "it is not activated through encoding='BOM'" *at all*.

However if this autodetection feature is useful in other cases (no
matter how it's activated), it should be a codec, because as part of the
open() function it isn't reusable.

>> The default value should provide the most useful
>> behaviour possible. Forcing users to choose between two different autodetection
>> strategies (encoding=None and another one) is a little insane IMO.

And encoding="mbcs" is a third option on Windows.

> That wouldn't disturb me much. There are a lot of things in that area
> that are a little insane, starting with Microsoft Windows :-)

;)

Servus,
   Walter


From barry at python.org  Mon Jan 11 13:37:46 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 07:37:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5DCE.3070909@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com> <4B4A5DCE.3070909@v.loewis.de>
Message-ID: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>

On Jan 10, 2010, at 6:07 PM, Martin v. L?wis wrote:

> As for decisions: I don't think there was an official BDFL pronouncent,
> but I recall Guido posting a message close to that, proposing that 2.7
> will be a release that will see bug fix releases for an indefinite
> period of time (where indefinite != infinite). This was shortly after
> him proposing that perhaps we shouldn't make a 2.7 release at all, and
> stop at 2.6.

IMO, the focus for Python 2.7 (and beyond) must be on helping people transition to Python 3.  That should be the reason why a 2.7 is released and, what should dictate whether a 2.8 is necessary.

Maybe everything people need (except manpower and round tuits) is already there.  I was pleasantly surprised to find that Distribute supports automatically running 2to3 at install time for Python packages.  This allowed me to "port" a recent (admittedly small) package to Python 3 by making it "python2.6 -3" clean and adding a couple of attributes to my setup.py[1].  I don't know how widely that feature's known, but I think it's *huge*, and exactly what we need to be doing.  The more we can make it easy for people to port their Python 2 code to Python 3, and to support that with one code base, the quicker we'll get to a predominantly Python 3 world.

So the question we should be asking is: what's missing from Python 2.7 to help with this transition?  If we can't get it into 2.7, then why, and would pushing it back to 2.8 help at all?

-Barry

[1] modulo a bug in Distribute that caused doctest in separate files to not be used when running  under Python 3.



From solipsis at pitrou.net  Mon Jan 11 13:39:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 13:39:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>  <4B4B1990.90705@livinglogic.de>
Message-ID: <1263213574.3327.0.camel@localhost>


> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

It is reusable as part of io.TextIOWrapper, though.

Regards

Antoine.




From regebro at gmail.com  Mon Jan 11 13:45:30 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 13:45:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
Message-ID: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>

On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

But an autodetect feature is not a codec. Sure it should be reusable,
but making it a codec seems to be  a weird hack to me. And how would
you reuse it if it was a codec? A reusable autodetect feature would be
useable to detect what codec it is. A autodetect codec would not be
useful for that, as it would simply just decode.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 14:21:15 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 14:21:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>	<4B4913DF.7050801@v.loewis.de>
	<4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B25CB.5030003@livinglogic.de>

On 11.01.10 13:45, Lennart Regebro wrote:

> On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
>> However if this autodetection feature is useful in other cases (no
>> matter how it's activated), it should be a codec, because as part of the
>> open() function it isn't reusable.
> 
> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

I think we already had this discussion two years ago in the context of
XML decoding ;):

http://mail.python.org/pipermail/python-dev/2007-November/075138.html

> And how would
> you reuse it if it was a codec? A reusable autodetect feature would be
> useable to detect what codec it is. A autodetect codec would not be
> useful for that, as it would simply just decode.

I have implemented an XML codec (as part of XIST:
http://pypi.python.org/pypi/ll-xist), that can do that:

>>> from ll import xml_codec
>>> import codecs
>>> c = codecs.getincrementaldecoder("xml")()
>>> c.encoding
>>> c.decode("<?xml")
u''
>>> c.encoding
>>> c.decode(" version='1.0'")
u''
>>> c.encoding
>>> c.decode(" encoding='iso-8859-1'?>")
u"<?xml version='1.0' encoding='iso-8859-1'?>"
>>> c.encoding
'iso-8859-1'

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 14:47:12 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 14:47:12 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B25CB.5030003@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B25CB.5030003@livinglogic.de>
Message-ID: <319e029f1001110547n1d1c9831p34b0eac519d13f9d@mail.gmail.com>

On Mon, Jan 11, 2010 at 14:21, Walter D?rwald <walter at livinglogic.de> wrote:
> I think we already had this discussion two years ago in the context of
> XML decoding ;):

Yup. Ans Martins answer then is my answer now:

"> So the code is good, if it is inside an XML parser, and it's bad if it
> is inside a codec?

Exactly so. This functionality just *isn't* a codec - there is no
encoding. Instead, it is an algorithm for *detecting* an encoding."

The conclusion was that a method do autodetect encodings would be
good. I think the same conclusion applies here.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 18:16:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:16:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<4B46F67F.60604@v.loewis.de>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B5CF5.50806@v.loewis.de>

> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

Well, the existing UTF-16 codec also is an autodetect feature (to
detect the endianness), and I don't consider it a weird hack.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 18:27:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:27:01 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B5CF5.50806@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B5CF5.50806@v.loewis.de>
Message-ID: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:16, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> But an autodetect feature is not a codec. Sure it should be reusable,
>> but making it a codec seems to be ?a weird hack to me.
>
> Well, the existing UTF-16 codec also is an autodetect feature (to
> detect the endianness), and I don't consider it a weird hack.

So the BOM codec should raise a UnicodeDecodeError if there is no BOM?
Because that's what it would have to do, in that case, because it
can't fall back on anything, it has to handle and implement all
encodings that have a BOM. And is it then actually very useful? You
would have to do a try/except first with encoding='BOM' and then
encoding=None to get the fallback to the standard.


I must say that I find this whole thing pretty obvious. 'BOM' is not
an encoding. Either there should be a method to get the encoding from
the BOM, returning None of there isn't one, or open() should look at
the BOM when you pass in encoding=None. Or both.

That covers all usecases, is easy and obvious. Either open(file=foo,
encoding=None) or open(file, encoding=encoding_from_bom(file))

I can't see that open(file, encoding='BOM') has any benefit over this,
covers any extra usecase and is clearer in any way. Instead it adds
something confusing: An encoding that isn't an encoding.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From python at mrabarnett.plus.com  Mon Jan 11 18:35:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:35:35 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<201001091438.43576.victor.stinner@haypocalc.com>	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <4B4B6167.40606@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
>> UTF-8 might be a good choice
> 
> No, fallback if there is no BOM should be the local settings, just as
> fallback is today if you don't specify a codec.
> I mean, what if you want to look for a BOM but fall back to something
> else? How far will we go with encoding special information in the
> codecs names? codec='BOM else UTF-16 else locale'? :-)
> 
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se. It should either be default to look for a
> BOM when codec=None, or a special parameter. If none of these are
> desired, then we need a special function that takes a filename or file
> handle, and looks for a BOM and returns the codec found or None. But
> I find that much less natural and obvious than checking the BOM when codec=None.
> 
Or pass a function that accepts a byte stream or the first few bytes and
returns the encoding and any unused bytes (because the byte stream might
not be seekable)?

def guess_encoding(byte_stream):
     data = byte_stream.read(2)
     if data == b"\xFE\xFF":
         return "UTF-16BE", b""
     return "UTF-8", data

text_file = open(filename, encoding=guess_encoding)
...


From python at mrabarnett.plus.com  Mon Jan 11 18:46:30 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:46:30 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
Message-ID: <4B4B63F6.1020309@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>> I know you are just looking for a compromise, but this shouldn't be
>> it: the PSF has deliberately stayed out of the actual Python
>> engineering, so the release that Benjamin makes is not done by the
>> PSF (but by Benjamin and his contributors :-).
> 
> Hm. Yeah. That's right of course. I started with saying "official", 
> but then I thought "official according to who?". :)

"Official" is whatever the BDFL says it is! :-)

> So I changed it to mentioning PSF, but that doesn't work either. I
> guess the current writing as as good as it gets, unless we change
> "scheduled" to "expected" or something.
> 


From martin at v.loewis.de  Mon Jan 11 18:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:59:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>	
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>	
	<4B4B5CF5.50806@v.loewis.de>
	<319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
Message-ID: <4B4B66EC.7000203@v.loewis.de>

> I must say that I find this whole thing pretty obvious. 'BOM' is not
> an encoding.

That I certainly agree with.

> That covers all usecases, is easy and obvious. Either open(file=foo,
> encoding=None) or open(file, encoding=encoding_from_bom(file))
> 
> I can't see that open(file, encoding='BOM') has any benefit over this,

Well, it would have the advantage that Walter pointed out: you can
implement it independent of the open() implementation, and even provide
it in older versions of Python.

Regards,
Martin



From regebro at gmail.com  Mon Jan 11 18:59:52 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:59:52 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B63F6.1020309@mrabarnett.plus.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
Message-ID: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:46, MRAB <python at mrabarnett.plus.com> wrote:
> "Official" is whatever the BDFL says it is! :-)

Heh, right.

So, it should say "Guido wants 2.7 to be the last main version of
Python 2, so it probably will be. We promise to release bugfixes it
for, like, ages".

No need to be needlessly formal. :-D
-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From olemis at gmail.com  Mon Jan 11 19:58:01 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 13:58:01 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
[...]
>

I had similar issues too (please read below ;o) ...

On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>

About guessing the encoding, I experienced this issue while I was
developing a Trac plugin. What I was doing is as follows :

- I guessed the MIME type + charset encoding using Trac MIME API (it
was a CSV file encoded using UTF-16)
- I read the file using `open`
- Then wrapped the file using `codecs.EncodedFile`
- Then used `csv.reader`

... and still get the BOM in the first value of the first row in the CSV file.

{{{
#!python

>>> mimetype
'utf-16-le'
>>> ef = EncodedFile(f, 'utf-8', mimetype)
}}}

IMO I think I am +1 for leaving `open` just like it is, and use module
`codecs` to deal with encodings, but I am strongly -1 for returning
the BOM while using `EncodedFile` (mainly because encoding is
explicitly supplied in ;o)

> --Guido
>

CMIIW anyway ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:


From mal at egenix.com  Mon Jan 11 21:44:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 11 Jan 2010 21:44:34 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
Message-ID: <4B4B8DB2.1060102@egenix.com>

Olemis Lang wrote:
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>> Hi,
>>>
>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>> BOM should be "ignored".
>>>
> [...]
>>
> 
> I had similar issues too (please read below ;o) ...
> 
> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
> 
> About guessing the encoding, I experienced this issue while I was
> developing a Trac plugin. What I was doing is as follows :
> 
> - I guessed the MIME type + charset encoding using Trac MIME API (it
> was a CSV file encoded using UTF-16)
> - I read the file using `open`
> - Then wrapped the file using `codecs.EncodedFile`
> - Then used `csv.reader`
> 
> ... and still get the BOM in the first value of the first row in the CSV file.

You didn't say, but I presume that the charset guessing logic
returned either 'utf-16-le' or 'utf-16-be' - those encodings don't
remove the leading BOM. The 'utf-16' codec will remove the BOM.

> {{{
> #!python
> 
>>>> mimetype
> 'utf-16-le'
>>>> ef = EncodedFile(f, 'utf-8', mimetype)
> }}}

Same here: the UTF-8 codec will not remove the BOM, you have
to use the 'utf-8-sig' codec for that.

> IMO I think I am +1 for leaving `open` just like it is, and use module
> `codecs` to deal with encodings, but I am strongly -1 for returning
> the BOM while using `EncodedFile` (mainly because encoding is
> explicitly supplied in ;o)

Note that EncodedFile() doesn't do any fancy BOM detection or
filtering. This is the job of the codecs.

Also note that BOM removal is only valid at the beginning of
a file. All subsequent BOM-bytes have to be read as-is (they
map to a zero-width non-breaking space) - without removing them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Mon Jan 11 22:12:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 07:12:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <4B4B9447.4060101@gmail.com>

Benjamin Peterson wrote:
 > My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
> 
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

I would call it a documentation bug: by leaving out the __get__ you're
telling Python to override *just* the setting of the attribute, while
still using the normal lookup process for retrieving the attribute (i.e.
allowing the attribute to be shadowed in the instance dictionary).

Adding a "print('Descriptor Invoked')" to the __set__ method in your
example:

>>> x = X()
>>> print(x.attr)
<__main__.Descr object at 0x7f283b51ac50>
>>> x.attr = 42
Descriptor invoked
>>> print(x.attr)
42
>>> x.attr = 6*9
Descriptor invoked
>>> print(x.attr)
54

Note that the behaviour here is still different from that of a data
descriptor: with a data descriptor, once it gets shadowed in the
instance dictionary, the descriptor is ignored *completely*. The only
way to get the descriptor involved again is to eliminate the shadowing.
The non-data descriptor with only __set__ is just choosing not to
override the attribute lookup process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From olemis at gmail.com  Mon Jan 11 22:29:38 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 16:29:38 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B8DB2.1060102@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
	<4B4B8DB2.1060102@egenix.com>
Message-ID: <24ea26601001111329i7f609aa1i9f5db7eb61f1fae7@mail.gmail.com>

Probably one part of this is OT , but I think it could complement the
discussion ;o)

On Mon, Jan 11, 2010 at 3:44 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Olemis Lang wrote:
>>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>>> <victor.stinner at haypocalc.com> wrote:
>>>> Hi,
>>>>
>>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>>> BOM should be "ignored".
>>>>
>> [...]
>>>
>>
>> I had similar issues too (please read below ;o) ...
>>
>> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>>>
>>
>> About guessing the encoding, I experienced this issue while I was
>> developing a Trac plugin. What I was doing is as follows :
>>
>> - I guessed the MIME type + charset encoding using Trac MIME API (it
>> was a CSV file encoded using UTF-16)
>> - I read the file using `open`
>> - Then wrapped the file using `codecs.EncodedFile`
>> - Then used `csv.reader`
>>
>> ... and still get the BOM in the first value of the first row in the CSV file.
>
> You didn't say, but I presume that the charset guessing logic
> returned either 'utf-16-le' or 'utf-16-be'

Yes. In fact they return the full mimetype 'text/csv; charset=utf-16-le' ... ;o)

> - those encodings don't
> remove the leading BOM.

... and they should ?

> The 'utf-16' codec will remove the BOM.
>

In this particular case there's nothing I can do, I have to process
whatever charset is detected in the input ;o)

>> {{{
>> #!python
>>
>>>>> mimetype
>> 'utf-16-le'
>>>>> ef = EncodedFile(f, 'utf-8', mimetype)
>> }}}
>
> Same here: the UTF-8 codec will not remove the BOM, you have
> to use the 'utf-8-sig' codec for that.
>
>> IMO I think I am +1 for leaving `open` just like it is, and use module
>> `codecs` to deal with encodings, but I am strongly -1 for returning
>> the BOM while using `EncodedFile` (mainly because encoding is
>> explicitly supplied in ;o)
>
> Note that EncodedFile() doesn't do any fancy BOM detection or
> filtering.

... directly.

> This is the job of the codecs.
>

OK ... to come back to the scope of the subject, in the general case,
I think that BOM (if any) should be handled by codecs, and therefore
indirectly by EncodedFile . If that's a explicit way of working with
encodings I'd prefer to use that wrapper explicitly in order to
(encode | decode) the file and let the codec detect whether there's a
BOM or not and ?adjust? `tell`, `read` and everything else in that
wrapper (instead of `open`).

> Also note that BOM removal is only valid at the beginning of
> a file. All subsequent BOM-bytes have to be read as-is (they
> map to a zero-width non-breaking space) - without removing them.
>

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Test cases for custom query (i.e report 9) ... PASS (1.0.0)  -
http://simelo.hg.sourceforge.net/hgweb/simelo/trac-gviz/rev/d276011e7297


From martin at v.loewis.de  Mon Jan 11 22:42:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:42:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
Message-ID: <4B4B9B55.1010709@v.loewis.de>

> So the question we should be asking is: what's missing from Python
> 2.7 to help with this transition?

Wrt. the distribute feature you've noticed: Python 3.0 already supports
that in distutils. So from day one of Python 3.x, you could have used
that approach for porting Python code. I had been promoting it ever
since, but it's taking up only slowly, partially because it has
competition in the approaches "burn the bridges" (i.e. convert to 3.x
once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
(i.e. try to write code that works in both 2.x and 3.x unmodified).

> If we can't get it into 2.7, then
> why, and would pushing it back to 2.8 help at all?

I've done a fair bit of 3.x porting, and I'm firmly convinced that
2.x can do nothing:

a) telling people that they have to move to 2.6 first actually
   hurts migration, instead of helping, because it implies to them
   that they have to drop old versions (e.g. 2.3.) - just because
   they had *always* dropped old versions before supporting new ones.
b) IMO, people also don't gain much by first migrating to 2.6.
   In principle, it gives them the opportunity to get py3k warnings.
   However, I haven't heard a single positive report where these
   warnings have actually helped people in porting. Yours is the
   first report saying that you followed the official guideline,
   but you didn't state whether doing so actually helped (or whether
   you just ported to 2.6 because the guideline told you to).
c) whatever 2.7 does (except perhaps for the warnings), it won't
   be useful to applications, because they couldn't use it, anyway:
   they need to support 2.4 and 2.5, and it won't have any of the
   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
   might actually hurt porting: people may have to special-case
   2.7 because their work-arounds for older versions may break in 2.7
   (e.g. testing whether a name is *not* defined, when it becomes
   defined in 2.7), plus it gives them an incentive to not port
   yet until they can drop support for anything before 2.7.

Inherently, 2.8 can't improve on that.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 22:44:24 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:44:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
Message-ID: <4B4B9BB8.3070505@v.loewis.de>

> So, it should say "Guido wants 2.7 to be the last main version of
> Python 2, so it probably will be. We promise to release bugfixes it
> for, like, ages".
> 
> No need to be needlessly formal. :-D

That summarizes my understanding of what Guido said fairly well :-)

+1 for putting it into the announcements.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Mon Jan 11 23:11:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 11 Jan 2010 22:11:44 +0000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4B9447.4060101@gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com>
Message-ID: <4B4BA220.20701@voidspace.org.uk>

On 11/01/2010 21:12, Nick Coghlan wrote:
> Benjamin Peterson wrote:
>   >  My question is: Is this a doc bug or a implementation bug? If the
>    
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>>      
> [snip...]
>
> Note that the behaviour here is still different from that of a data
> descriptor: with a data descriptor, once it gets shadowed in the
> instance dictionary, the descriptor is ignored *completely*. The only
> way to get the descriptor involved again is to eliminate the shadowing.
> The non-data descriptor with only __set__ is just choosing not to
> override the attribute lookup process.
>
>    

Does that mean we need a third class of descriptors that are neither 
data descriptors nor non-data descriptors?

What should we call them: really-not-data-descriptors?

All the best,

Michael

> Cheers,
> Nick.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From guido at python.org  Mon Jan 11 23:55:01 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Jan 2010 14:55:01 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9BB8.3070505@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de> 
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com> 
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com> 
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com> 
	<4B4B9BB8.3070505@v.loewis.de>
Message-ID: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>

+1 from me. (With the caveat that "time will tell" is definitely the
ultimate answer. Plans may change unexpectedly, even though we don't
expect them to.)

PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
helpful. But I trust he has ported a lot more code to 3.x than I have
personally. Are there other experiences?

--Guido

On Mon, Jan 11, 2010 at 1:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So, it should say "Guido wants 2.7 to be the last main version of
>> Python 2, so it probably will be. We promise to release bugfixes it
>> for, like, ages".
>>
>> No need to be needlessly formal. :-D
>
> That summarizes my understanding of what Guido said fairly well :-)
>
> +1 for putting it into the announcements.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Tue Jan 12 00:16:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 00:16:47 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <4B4BB15F.5020807@v.loewis.de>

Guido van Rossum wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
> 
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. 

That's really because I haven't even attempted to use it. Instead, the
software would fall flat on its own when running the test suite in 3.x.
IOW, I don't need 2.6 to tell me what might break when I can see 3.x
actually breaking.

Regards,
Martin


From david.lyon at preisshare.net  Tue Jan 12 00:22:42 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:22:42 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>


Hi Martin,

> Of course, the less active fraction of Python contributors may not
> notice, since they just chose to not contribute (which, of course,
> is fine). However, asking me to work twice as much as I want to
> on the project to keep two branches alive is just unfair.

Totally true. Actually as an end-developer I'd say that python
2.x series from a programming perspective is quite good. It
doesn't need the addition of a lot of new features imho.

So for me, I think too much time spent there would be not
yield great benefits.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Well, we all know your work is super quality. :-) That's not
being contested.

However, Quality can be measured different ways and it can
be assessed in different ways. Quality itself is a subjective
thing.

The point I'm only making is that if a piece of software
doesn't have "new" things added over time, then users
can get a reverse impression of a lack of quality.

We've all seen where 'internal' quality can increase
and user perceptions can decrease.

It could be things like improved graphics and things readily
apparent to the user.

At the moment, I would say that the "internal" quality
of the python 2.x series is super high. "external" quality
issues such as the packaging dilemma give the user the
opposite quality experience. But people are working on
that as best they can elsewhere. I'll leave it at that.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Python 3.x needs more carrots.

>From an ordinary (perphaps ignorant) user perspective there
is nothing. Yes, we know if we actually will start
programming then we will like it more.

But my wishes to Santa Claus would be allow the free
flow of PEPs for Python 3 packaging. Even encourage
it.

As an end developer, here's what I'd like from Santa
in 2010 to get me to swap to python 3:

 * get all the packages on pypi tested for python 3

 * put a web based package manager in python 3. This
   would perhaps be based around PIP (keep many
   people happy) and would look much like the built
   in web-console that you get with a $200 router.

 * Incorporate SCM based end-user software installs
   by adding to python3-stdlib the SCM support
   packages (cvs,bzr,svn,hg). This would *really*
   help in the scientific community.

 * put a web interface on distutils also so that
   we don't have to use a command line interface.
   (I want a pic of a smiley girl to great me
    when I build something - "Are you ready
    to build now Honey?"). ok - I joke. But the
    point is made.

So, ok, maybe these things aren't about 'code'
quality. But rather user experience.

Things like these do count also as "quality"
via the technical term "perception of quality".

If the PEP process is as unblocked as the
documentation implies, implying that anybody
can contribute to Python 3. Then there shouldn't
be any issue.

David









From solipsis at pitrou.net  Tue Jan 12 00:37:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 23:37:47 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100112T003506-611@post.gmane.org>

David Lyon <david.lyon <at> preisshare.net> writes:
> 
> > This has nothing to do with pushing 3.x, but all with managing
> > available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As someone who experiences the difference almost every day, I can say 3.x
definitely has enough carrots for me. The simple fact that I will be able to
raise exceptions with non-ASCII error messages without having to workaround a
hopelessly broken conversion/reporting logic is a huge improvement. And that's
only a small part of the picture.

Regards

Antoine.




From janssen at parc.com  Tue Jan 12 00:47:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 11 Jan 2010 15:47:55 PST
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <17215.1263253675@parc.com>

> David Lyon <david.lyon <at> preisshare.net> writes:
> > 
> > > This has nothing to do with pushing 3.x, but all with managing
> > > available manpower and still providing quality software.
> > 
> > Python 3.x needs more carrots.

I'd be happy to move UpLib to Python 3, when the various packages that I
need are also on Python 3.  And that depresses me to think about.  I
need

   Medusa
   ReportLab
   PyLucene
   Email
   Vobject
   Mutagen
   Pyglet
   Hachoir
   Win32

I'm on the mailing lists for a lot of these, and the only one that I
know is thinking about Python 3 is Email.  I'd expect Win32 and PIL to
also be thinking about it, but I haven't heard anything.

Bill


From david.lyon at preisshare.net  Tue Jan 12 00:47:51 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:47:51 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <1220.218.214.45.58.1263253671.squirrel@syd-srv02.ezyreg.com>

Antoine writes:

> As someone who experiences the difference almost every day, I can say 3.x
> definitely has enough carrots for me. The simple fact that I will be able
> to
> raise exceptions with non-ASCII error messages without having to
> workaround a
> hopelessly broken conversion/reporting logic is a huge improvement. And
> that's
> only a small part of the picture.

In no way could I disagree with you.

Ascii works fine for us here - but I guess we're lucky.

Just keep pushing python 3 and have a nice day..

David





From barry at python.org  Tue Jan 12 01:11:28 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:11:28 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100111191128.39020d89@freewill>

On Jan 11, 2010, at 10:42 PM, Martin v. L?wis wrote:

>Wrt. the distribute feature you've noticed: Python 3.0 already supports
>that in distutils. So from day one of Python 3.x, you could have used
>that approach for porting Python code. I had been promoting it ever
>since, but it's taking up only slowly, partially because it has
>competition in the approaches "burn the bridges" (i.e. convert to 3.x
>once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
>(i.e. try to write code that works in both 2.x and 3.x unmodified).

Interesting.  The only reason I never used it before is because I didn't know
about it.  Am I alone?

Maybe I'm projecting my own preferences, but it seems to me that supporting
both Python 2 and 3 from the same code base would be a very powerful way to
enable a large amount of existing code to support Python 3.  I'm certainly
going to do this from now on with all the libraries I maintain.  I don't have
any cycles to write code twice and I can't abandon Python 2 yet.  I'm
skeptical that code can work unmodified in both 2 and 3 without 2to3.

As an example, the one library I've already ported used a metaclass.  I don't
see any way to specify that the metaclass should be used in a portable way.
In Python 2.6 it's:

class Foo:
    __metaclass__ = Meta

and in Python 3 it's:

class Foo(metaclass=Meta):

2to3 made that pain go away.

>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>2.x can do nothing:
>
>a) telling people that they have to move to 2.6 first actually
>   hurts migration, instead of helping, because it implies to them
>   that they have to drop old versions (e.g. 2.3.) - just because
>   they had *always* dropped old versions before supporting new ones.

Is it just an implication, or is it reality?  I have yet to try to support
older versions of Python and also support Python 3.  (The one package I've
done this with so far only supports Python 2.6.)

>b) IMO, people also don't gain much by first migrating to 2.6.
>   In principle, it gives them the opportunity to get py3k warnings.
>   However, I haven't heard a single positive report where these
>   warnings have actually helped people in porting. Yours is the
>   first report saying that you followed the official guideline,
>   but you didn't state whether doing so actually helped (or whether
>   you just ported to 2.6 because the guideline told you to).

Python 2.6 has other useful features, which I want to take advantage of, so
getting py3k warnings is a bonus.  Running 'python2.6 -3 setup.py test' over
my code did in fact help clean up a couple of problems.  Seems like a good
first step when considering Python 3 support to me.

>c) whatever 2.7 does (except perhaps for the warnings), it won't
>   be useful to applications, because they couldn't use it, anyway:
>   they need to support 2.4 and 2.5, and it won't have any of the
>   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
>   might actually hurt porting: people may have to special-case
>   2.7 because their work-arounds for older versions may break in 2.7
>   (e.g. testing whether a name is *not* defined, when it becomes
>   defined in 2.7), plus it gives them an incentive to not port
>   yet until they can drop support for anything before 2.7.
>
>Inherently, 2.8 can't improve on that.

I'm not so sure.  Yes, as a package maintainer there are older versions to
think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
released, what will be the minimum version of Python provided by most OS
vendors (where the majority of Python users probably get their 'python')?  I
guess some people will have to support everything from Python 2.3 to 2.8 but
you're talking supporting something like a spread of 7 years of Python
versions.  What other platform do you support for 7 years?  Even Ubuntu long
term support (LTS) releases only live for 5 years and even then, newer Pythons
are often back ported.  Or if not, then who cares?  You're not going to use a
newer version of a library on an LTS anyway.

I'm not necessarily saying that justifies a 2.8 release.  I'm just asking how
we can make it easier for people to port to Python 3.  The automatic 2to3 has
already made it easier for me to port one library to Python 3 because I barely
had to even think about it.  Maybe that's enough.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/840169eb/attachment-0004.pgp>

From barry at python.org  Tue Jan 12 01:12:19 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:12:19 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <20100111191219.5fdd2542@freewill>

On Jan 11, 2010, at 02:55 PM, Guido van Rossum wrote:

>PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
>helpful. But I trust he has ported a lot more code to 3.x than I have
>personally. Are there other experiences?

I've only done one small library so far, but it was helpful to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/ba78a57a/attachment-0004.pgp>

From andrew at bemusement.org  Tue Jan 12 01:07:26 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 12 Jan 2010 11:07:26 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100112000726.GO32351@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> I've done a fair bit of 3.x porting, and I'm firmly convinced that
> 2.x can do nothing:
[...]
> Inherently, 2.8 can't improve on that.

I agree that there are limitations like the ones you've listed, but I
disagree with your conclusion.  Maybe you assume that it's just as hard
to move to 2.8 (using the py3k backports not available in say 2.5) as it
is to 3.x?

But a hypothetical 2.8 would also give people a way to move closer to
py3k without giving up on using all their 2.x-only dependencies.  I
think it's much more likely that libraries like Twisted can support 2.8
in the near future than 3.x.

Then, when all of your dependencies (or viable alternatives to those
dependencies) are available for 3.x, you'll have an easier transition if
you can start from a 2.x with fewer differences in features.

Fundamentally the more 2.x can converge on 3.x, the easier it will be
for users to make the leap, because it will be a smaller leap.  The
longer the 2.x series lives, the more these newer 2.x versions like 2.7
and maybe even 2.8 will be available on common platforms for people to
depend upon as minimum versions, which means that as time goes by they
can depend on a version that's closer to 3.x.  And so again, the leap
becomes easier to make.  So to me it's pretty clear that 2.8 *can*
improve the transition path to 3.x.  It may or may not be a worthwhile
use of effort for python-dev to make 2.8, but that's different to saying
it's inherently pointless.

-Andrew.


From solipsis at pitrou.net  Tue Jan 12 01:28:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 00:28:26 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <loom.20100112T012550-387@post.gmane.org>

Andrew Bennetts <andrew <at> bemusement.org> writes:
> 
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies.  I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

I don't think a 2.8 could make you much closer to 3.x. AFAICT, every possible
way to ease transition to 3.x will already be in 2.7, or almost. But perhaps I'm
lacking imagination.

Regards

Antoine.




From brian.curtin at gmail.com  Tue Jan 12 02:57:46 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 11 Jan 2010 19:57:46 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>

On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:

> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
>
> -Brett
>
I think it would be interesting to see how people are using the tracker, or
how they want to be using it. For example, there are currently over 1500
open issues with no stage set, some of which seemingly haven't been read by
anyone at all. Would a properly set stage field save issues from falling
into a black hole?

Food for thought: according to the last tracker summary, there are over 1000
open issues with a patch, and issues stay open an average of 700 days
(median 450).

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/a3439436/attachment-0004.htm>

From jackdied at gmail.com  Tue Jan 12 04:53:24 2010
From: jackdied at gmail.com (Jack Diederich)
Date: Mon, 11 Jan 2010 22:53:24 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>

On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

[sidebar]
1) the metaclass fixer was a PITA to implement.
2) 95% of __metaclass__ definitions searchable via google code were of
the "__metaclass__ = type" variety.  The 2to3 patch exists only
because of the few other uses.
3) 100% of the module level assignments in public projects were the
"__metaclass__ = type" variety which is why there isn't a fixer for
that.  Also, a fixer would have been really, really ugly (munge every
class definition in this module because there is a top level
assignment).

-Jack


From benjamin at python.org  Tue Jan 12 05:01:40 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 Jan 2010 22:01:40 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <1afaf6161001112001t5e7bab83t7d93f33fa11ce849@mail.gmail.com>

2010/1/11 Jack Diederich <jackdied at gmail.com>:
> On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
>> As an example, the one library I've already ported used a metaclass. ?I don't
>> see any way to specify that the metaclass should be used in a portable way.
>> In Python 2.6 it's:
>>
>> class Foo:
>> ? ?__metaclass__ = Meta
>>
>> and in Python 3 it's:
>>
>> class Foo(metaclass=Meta):
>>
>> 2to3 made that pain go away.
>
> [sidebar]
> 1) the metaclass fixer was a PITA to implement.

Does this make it any less useful, though? :)



-- 
Regards,
Benjamin


From steven.bethard at gmail.com  Tue Jan 12 06:57:18 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 11 Jan 2010 21:57:18 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>

On Mon, Jan 11, 2010 at 4:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

Actually there's a solution to this one too:

    FooBase = Meta('FooBase', (), {})
    class Foo(FooBase):
        ...

That should work in Python 2.X and 3.X.

I've got argparse running on Python 2.3-3.1, and the changes were
pretty easy. You can see them all in the revision here:

    http://code.google.com/p/argparse/source/detail?r=12

I have aspirations of putting all of the tricks I learned up up on the
Wiki somewhere, but I just haven't had the time.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From regebro at gmail.com  Tue Jan 12 07:30:10 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:30:10 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>

On Mon, Jan 11, 2010 at 23:55, Guido van Rossum <guido at python.org> wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
>
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. But I trust he has ported a lot more code to 3.x than I have
> personally. Are there other experiences?

It doesn't warn for that many of the unportable problems, but I'm not
sure it can warn for them either. It's certainly helpful, just not
very much. :) I think the biggest help was added in 2.6.2, and that's
warning for old integer division. It will also warn for modules that
aren't supported anymore, if I remember correctly, and that's often
helpful. But it won't warn for real tricky problems, like binary vs
text in strings, and I don't see how it can either.

And, I just realized, it doesn't warn for you using cmp or __cmp__
either, and 2to3 won't fix that, so it should actually warn for it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:49:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:49:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:11, Barry Warsaw <barry at python.org> wrote:
> Interesting. ?The only reason I never used it before is because I didn't know
> about it. ?Am I alone?

Maybe not, but the Distribute feature is there because IMO the
distutils feature by itself isn't particularily useful. You need to
write your own distutils extensions, in practice, and they are not
trivial. Distribute has simply done it for you. :)

> I'm skeptical that code can work unmodified in both 2 and 3 without 2to3.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:53:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:53:20 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <319e029f1001112253x511f8109ja2300a022010ef99@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:07, Andrew Bennetts <andrew at bemusement.org> wrote:
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. ?I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

When 2.7 was discussed several people agreed that 2.7 should include
as much 3.x stuff as possible to ease transition. That turned out to
not be very much, so I'm not sure there is more. :)

Unless of course, 2.8 starts including more of the refactored
libraries, but that's a very minor issue in porting, so it won't help
much.

To really help, it needs to start implement things that break
backwards compatibility. That would be weird. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ncoghlan at gmail.com  Tue Jan 12 10:39:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:39:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4BA220.20701@voidspace.org.uk>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com> <4B4BA220.20701@voidspace.org.uk>
Message-ID: <4B4C435B.7080903@gmail.com>

Michael Foord wrote:
>> Note that the behaviour here is still different from that of a data
>> descriptor: with a data descriptor, once it gets shadowed in the
>> instance dictionary, the descriptor is ignored *completely*. The only
>> way to get the descriptor involved again is to eliminate the shadowing.
>> The non-data descriptor with only __set__ is just choosing not to
>> override the attribute lookup process.
>>
>>    
> 
> Does that mean we need a third class of descriptors that are neither
> data descriptors nor non-data descriptors?

Not really - leaving out __get__ when defining __set__ just creates a
data descriptor that happens to use the default attribute look-up
process rather than defining a different one.

(Note that I had the data/non-data terminology backwards in my previous
message - I tend to think of the two kinds of descriptor as enforced and
non-enforced respectively precisely because I have to think about it to
remember that "functions are non-data descriptors and properties are
data descriptors, hence non-data descriptors only define __get__ while
data descriptors define __set__". I don't find the data/non-data
terminology intuitive at all)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:44:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:44:18 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>	<4B4B9BB8.3070505@v.loewis.de>	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
	<319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
Message-ID: <4B4C4472.10104@gmail.com>

Lennart Regebro wrote:
> And, I just realized, it doesn't warn for you using cmp or __cmp__
> either, and 2to3 won't fix that, so it should actually warn for it.

I have a vague recollection that we tried to warn for that and ended up
nixing the warning due to vast swarms of false alarms (or because you
couldn't write correct 2.x code without triggering it).

I'd be happy for someone to prove my recollection wrong though (i.e. by
writing a patch that implements such warnings).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:48:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:48:57 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4C4589.70906@gmail.com>

David Lyon wrote:
>> This has nothing to do with pushing 3.x, but all with managing
>> available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As Guido has said a few times, the gains are far greater for *new*
Python developers than they are for existing ones.

Existing developers have to unlearn old habits and wait for libraries
they already use to get ported. New developers can just get started with
a much cleaner language. They don't have as rich a 3rd party ecosystem
on Python 3 as they would on Python 2.x at this point in time, but
unlike existing developers they also have the luxury of cherry-picking
just those packages that already have Python 3 support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Jan 12 11:20:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:20:27 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <hihida$unc$1@ger.gmane.org>

Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit?:
> 
> For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all.

I think most issues /have/ been read. It's just that for many of them, 
nobody is interested enough in or feels competent enough for fixing them.

> Would a properly set stage field save issues from
> falling into a black hole?

I don't think this has anything to do with properly setting the stage 
field. We just have limited time and manpower. Perhaps one of our goals 
should be to reach out more to potential contributors.

> Food for thought: according to the last tracker summary, there are over
> 1000 open issues with a patch, and issues stay open an average of 700
> days (median 450).

As for open issues with a patch, a "code review" button sending this 
patch to Rietveld (or any other similar tool) is something many of us 
have been hoping for :-) It would make reviewing easier, faster and more 
thorough (because you visualize things better than by looking at a diff).

Regards

Antoine.



From solipsis at pitrou.net  Tue Jan 12 11:36:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:36:49 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <hihjc1$45m$1@ger.gmane.org>

Le Tue, 12 Jan 2010 10:20:27 +0000, Antoine Pitrou a ?crit?:
> 
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.

Speaking of which, Steve had something to say about it:
http://holdenweb.blogspot.com/2009/11/comments-or-not-public-or-
private.html

cheers

Antoine.



From ncoghlan at gmail.com  Tue Jan 12 13:10:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 22:10:14 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <4B4C66A6.2040601@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
>> For example, there are currently over
>> 1500 open issues with no stage set, some of which seemingly haven't been
>> read by anyone at all.
> 
> I think most issues /have/ been read. It's just that for many of them, 
> nobody is interested enough in or feels competent enough for fixing them.

There are actually a whole host of reasons issues can stagnate:
- a feature request may seem reasonable (hence it doesn't get rejected
outright), but the right API may not be clear (hence it doesn't get
implemented in the near term)
- a patch may be reviewed and found to have significant defects (or
simply be overreaching the stated goal) but the original patch poster
loses interest after meeting resistance in their ambition to fix
something that is "obviously" broken
- a problem may simply be hard to fix in a backwards compatible way (or
even at all!)

Ultimately it boils down to Antoine's two categories (lack of interest
and lack of relevant expertise to make a final decision) but there are a
lot of different ways to get into those two buckets.

Because we aren't ruthless about pruning those kinds of issues out of
the tracker they're the ones that are going to accumulate over time.

I'd actually be interested to know what the issue stats are like when
RFEs are excluded and when the search is limited to the items flagged as
'easy'. If easy bug fix issues are taking a long time to get closed than
that would bother me a lot more than RFEs that sit on the tracker for years.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 12 13:14:47 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:14:47 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <20100112071447.675c8f24@freewill>

On Jan 11, 2010, at 10:53 PM, Jack Diederich wrote:

>3) 100% of the module level assignments in public projects were the
>"__metaclass__ = type" variety which is why there isn't a fixer for
>that.  Also, a fixer would have been really, really ugly (munge every
>class definition in this module because there is a top level
>assignment).

And almost certainly unnecessary.  IME, those are all to easily make bare
class definitions new style in Python 2.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/29b5ff24/attachment-0004.pgp>

From barry at python.org  Tue Jan 12 13:16:09 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:16:09 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
Message-ID: <20100112071609.1dcfffa6@freewill>

On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:

>Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
>That should work in Python 2.X and 3.X.

Ugly, but good call! :)

>I've got argparse running on Python 2.3-3.1, and the changes were
>pretty easy. You can see them all in the revision here:
>
>    http://code.google.com/p/argparse/source/detail?r=12
>
>I have aspirations of putting all of the tricks I learned up up on the
>Wiki somewhere, but I just haven't had the time.

The more resources we can provide people, both in code and in documentation,
the better.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/a2ba5b3e/attachment-0004.pgp>

From fuzzyman at voidspace.org.uk  Tue Jan 12 13:29:12 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 12:29:12 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112071609.1dcfffa6@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
	<20100112071609.1dcfffa6@freewill>
Message-ID: <4B4C6B18.7050008@voidspace.org.uk>

On 12/01/2010 12:16, Barry Warsaw wrote:
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>    
>> Actually there's a solution to this one too:
>>
>>     FooBase = Meta('FooBase', (), {})
>>     class Foo(FooBase):
>>         ...
>>
>> That should work in Python 2.X and 3.X.
>>      
> Ugly, but good call! :)
>
>    

There are all sorts of tricks. For example you can do exception handling 
that works with pre-2.6 syntax and 3.0 with a bare except and using 
sys.exc_info. It is horrible, but acceptable for short pieces of code (I 
have a couple of small modules that do this).

I haven't yet tried converting larger code-bases to Python 3, but I 
think the workflow advocated by Martin is greatly preferable to the 
hacks and tricks needed to make the same codebase run under 2 & 3.

Michael

>> I've got argparse running on Python 2.3-3.1, and the changes were
>> pretty easy. You can see them all in the revision here:
>>
>>     http://code.google.com/p/argparse/source/detail?r=12
>>
>> I have aspirations of putting all of the tricks I learned up up on the
>> Wiki somewhere, but I just haven't had the time.
>>      
> The more resources we can provide people, both in code and in documentation,
> the better.
>
> Thanks!
> -Barry
>    
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/7acd54f3/attachment-0004.htm>

From mal at egenix.com  Tue Jan 12 14:09:50 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 12 Jan 2010 14:09:50 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <4B4C749E.4040009@egenix.com>

Brett Cannon wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> 
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> * argparse (PEP 389)
> * brief mention on still wanting to break out the stdlib from CPython
> * an official policy on extension modules? (i.e. must have a pure Python
> implementation which can mean a ctypes implementation unless given an
> explicit waiver)
> 
> That's everything from a stdlib perspective. I have half-baked ideas, but
> nothing concrete enough to bring up unless people really want to discuss
> stuff like how to potentially standardize module deprecation warnings, etc.
> 
> In terms of me personally, I do plan to bring up at some point during the
> summit these points which don't squarely fit during my time slot:
> 
> * an official unofficial policy on how new proposed features should come to
> us (i.e. working code to python-ideas with a shell of a PEP that includes
> abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
> 
> If there is something missing from the stdlib discussion that you think
> should be brought up at the summit let me know. And if there is something
> here you want to discuss before the summit let me know and I can start a
> separate thread on it.

Could you please put these things and the results up on the Python
wiki ?!

We're going to have a language summit at EuroPython this year
as well and may want to continue/extend the discussion based
on what you're doing at PyCon.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From brian.curtin at gmail.com  Tue Jan 12 15:33:06 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 12 Jan 2010 08:33:06 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <cf9f31f21001120633n6460b55as6064228cce41c949@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:20, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
> >
> > For example, there are currently over
> > 1500 open issues with no stage set, some of which seemingly haven't been
> > read by anyone at all.
>
> I think most issues /have/ been read. It's just that for many of them,
> nobody is interested enough in or feels competent enough for fixing them.
>
> > Would a properly set stage field save issues from
> > falling into a black hole?
>
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.


Agreed, I didn't mean to place blame on the stage field, I just ran with how
I view that field since it was mentioned. When I'm thinking in "code-writing
developer mode" (tm), I'm more likely to go to issues which have a stage so
I know what I'm going into -- what needs to be worked on. When I'm in
project cleanup mode, I go by stageless issues -- what is necessary for this
to begin or end work. Maybe others work differently...software projects take
all kinds :-)

Anyways, sorry for the off-topic. If this is a summit worthy discussion,
cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/cf466340/attachment-0004.htm>

From rdmurray at bitdance.com  Tue Jan 12 17:12:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 11:12:28 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C66A6.2040601@gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org> <4B4C66A6.2040601@gmail.com>
Message-ID: <20100112161228.300EA1D688D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 22:10:14 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There are actually a whole host of reasons issues can stagnate:
> - a feature request may seem reasonable (hence it doesn't get rejected
> outright), but the right API may not be clear (hence it doesn't get
> implemented in the near term)
> - a patch may be reviewed and found to have significant defects (or
> simply be overreaching the stated goal) but the original patch poster
> loses interest after meeting resistance in their ambition to fix
> something that is "obviously" broken
> - a problem may simply be hard to fix in a backwards compatible way (or
> even at all!)

I would add:

- a patch is reviewed but the reviewer requests a second opinion before
  committing, which never arrives, and the original reviewer forgets
  about the issue.

I have occasionally observed apparently moribund issues spring to life
and get resolved when a triage person does something as simple as updating
the releases to which an issue applies.

> Ultimately it boils down to Antoine's two categories (lack of interest
> and lack of relevant expertise to make a final decision) but there are a
> lot of different ways to get into those two buckets.

There's a third bucket here: lack of time.  Sometimes there is interest
and expertise, but no time.  Worse, sometimes we end up waiting on the
person with the expertise and interest but no time, when someone else
with somewhat less expertise but more time should just go ahead and do
the commit.  (*That* one should be fixable.)

> Because we aren't ruthless about pruning those kinds of issues out of
> the tracker they're the ones that are going to accumulate over time.

Another other category that hangs around in the tracker is items for
which there simply isn't a consensus.  I suppose those could be brought
to python-dev for a final decision if someone wanted to tackle that task.

And I don't think it is a lack of ruthlessness.  I think it is the current
community consensus that we leave these issues open in the tracker in
case someone does come along and want to deal with them. (*)

> I'd actually be interested to know what the issue stats are like when
> RFEs are excluded and when the search is limited to the items flagged as
> 'easy'. If easy bug fix issues are taking a long time to get closed than
> that would bother me a lot more than RFEs that sit on the tracker for
> years.

I'd also be interested to know what the average and median lifetime
of closed bug reports is.  Not the metrics for open issues, but the
metrics for closed ones.  My theory is that we close a lot of bugs fairly
promptly, but the ones in the above categories make the average age of
*open* issues high.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls

(*) For example, I'm currently going through all the open issues
relating to the email package, and some of them are pretty darn
old, yet still have useful content.


From brett at python.org  Tue Jan 12 18:40:13 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:40:13 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C749E.4040009@egenix.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<4B4C749E.4040009@egenix.com>
Message-ID: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>

On Tue, Jan 12, 2010 at 05:09, M.-A. Lemburg <mal at egenix.com> wrote:

> Brett Cannon wrote:
> > Michael has given me the hg transition/stdlib time slot at the language
> > summit this year. In regards to that I plan to lead a discussion on:
> >
> > * where we are at w/ the Hg transition (Dirkjan should be there and I did
> a
> > blog post on this topic recently:
> > http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> > * argparse (PEP 389)
> > * brief mention on still wanting to break out the stdlib from CPython
> > * an official policy on extension modules? (i.e. must have a pure Python
> > implementation which can mean a ctypes implementation unless given an
> > explicit waiver)
> >
> > That's everything from a stdlib perspective. I have half-baked ideas, but
> > nothing concrete enough to bring up unless people really want to discuss
> > stuff like how to potentially standardize module deprecation warnings,
> etc.
> >
> > In terms of me personally, I do plan to bring up at some point during the
> > summit these points which don't squarely fit during my time slot:
> >
> > * an official unofficial policy on how new proposed features should come
> to
> > us (i.e. working code to python-ideas with a shell of a PEP that includes
> > abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> > * any changes needed to the issue tracker to help with the workflow?
> (stage
> > field seems like a failed experiment and we now have several effective
> > triage people who can help w/ guiding changes)
> >
> > If there is something missing from the stdlib discussion that you think
> > should be brought up at the summit let me know. And if there is something
> > here you want to discuss before the summit let me know and I can start a
> > separate thread on it.
>
> Could you please put these things and the results up on the Python
> wiki ?!
>
> We're going to have a language summit at EuroPython this year
> as well and may want to continue/extend the discussion based
> on what you're doing at PyCon.
>
>
I expect there will be at least summary emails on what gets discussed. There
is also a chance that it will be videotaped.

-Brett


> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 12 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/04964610/attachment-0004.htm>

From brett at python.org  Tue Jan 12 18:47:50 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:47:50 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>

On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
>
>>  * any changes needed to the issue tracker to help with the workflow?
>> (stage field seems like a failed experiment and we now have several
>> effective triage people who can help w/ guiding changes)
>>
>> -Brett
>>
> I think it would be interesting to see how people are using the tracker, or
> how they want to be using it. For example, there are currently over 1500
> open issues with no stage set, some of which seemingly haven't been read by
> anyone at all. Would a properly set stage field save issues from falling
> into a black hole?
>
>
"Using" is the key word there. I know this thread went on about how bugs
tend to end up being left open, but what I was proposing to talk about was
whether there is any shift desired by the seasoned tracker users to help in
their work. For instance, the Stage field was meant to help, but I don't
think it is really getting used much, which makes it at best just a UI junk
and at worst something to confuse new users. So I just wanted to discuss
things as a group to see if we could streamline some things, add others,
etc.

At worst I will try to grab people like Mark D., R. David, Antoine, Georg,
etc. at the summit (not sure exactly which the heavy bug closers will be
there), take them aside, and flat-out ask what they need to make their jobs
easier.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/3f808530/attachment-0004.htm>

From brett at python.org  Tue Jan 12 19:29:06 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:29:06 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C6B18.7050008@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com> 
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org> 
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill> 
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com> 
	<20100112071609.1dcfffa6@freewill> <4B4C6B18.7050008@voidspace.org.uk>
Message-ID: <bbaeab101001121029v28a863ccg8213ed494f15160c@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 12/01/2010 12:16, Barry Warsaw wrote:
>
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>
>
>  Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
> That should work in Python 2.X and 3.X.
>
>
>  Ugly, but good call! :)
>
>
>
>
> There are all sorts of tricks. For example you can do exception handling
> that works with pre-2.6 syntax and 3.0 with a bare except and using
> sys.exc_info. It is horrible, but acceptable for short pieces of code (I
> have a couple of small modules that do this).
>
> I haven't yet tried converting larger code-bases to Python 3, but I think
> the workflow advocated by Martin is greatly preferable to the hacks and
> tricks needed to make the same codebase run under 2 & 3.
>
>
In other words we need to pull together a HOWTO for Python source like the
one for extension modules that Benjamin wrote and make it rather prominently
linked from the Python 3 documentation index page.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/86cc3a21/attachment-0004.htm>

From rdmurray at bitdance.com  Tue Jan 12 19:31:23 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 13:31:23 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
Message-ID: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> >>  * any changes needed to the issue tracker to help with the workflow?
> >> (stage field seems like a failed experiment and we now have several
> >> effective triage people who can help w/ guiding changes)
> >>
> > I think it would be interesting to see how people are using the tracker, or
> > how they want to be using it. For example, there are currently over 1500
> > open issues with no stage set, some of which seemingly haven't been read by
> > anyone at all. Would a properly set stage field save issues from falling
> > into a black hole?
> >
> "Using" is the key word there. I know this thread went on about how bugs
> tend to end up being left open, but what I was proposing to talk about was
> whether there is any shift desired by the seasoned tracker users to help in
> their work. For instance, the Stage field was meant to help, but I don't
> think it is really getting used much, which makes it at best just a UI junk
> and at worst something to confuse new users. So I just wanted to discuss
> things as a group to see if we could streamline some things, add others,
> etc.

Well, I for one find the stage field useful.  It reminds me at a glance
where the issue is at in the workflow (and 'not set' is a valid place
in the workflow that an issue sometimes stays in for a while for reason
other than not having been triaged yet).  I suspect that if we discussed
it as a group (we who make the most changes on the tracker) we could
improve it, and the interface in general.

By the way, you could talk to people who aren't going to be at the
summit on #python-dev; I think all the currently tracker-active people
hang out there on a regular basis.

I'll have to give some thought to what changes/improvements might be
most useful, now that I've been doing this for a while.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Tue Jan 12 19:34:02 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:34:02 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com> 
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com> 
	<20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
Message-ID: <bbaeab101001121034m6de86385u4e0e72301e404a59@mail.gmail.com>

On Tue, Jan 12, 2010 at 10:31, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> > On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> > >>  * any changes needed to the issue tracker to help with the workflow?
> > >> (stage field seems like a failed experiment and we now have several
> > >> effective triage people who can help w/ guiding changes)
> > >>
> > > I think it would be interesting to see how people are using the
> tracker, or
> > > how they want to be using it. For example, there are currently over
> 1500
> > > open issues with no stage set, some of which seemingly haven't been
> read by
> > > anyone at all. Would a properly set stage field save issues from
> falling
> > > into a black hole?
> > >
> > "Using" is the key word there. I know this thread went on about how bugs
> > tend to end up being left open, but what I was proposing to talk about
> was
> > whether there is any shift desired by the seasoned tracker users to help
> in
> > their work. For instance, the Stage field was meant to help, but I don't
> > think it is really getting used much, which makes it at best just a UI
> junk
> > and at worst something to confuse new users. So I just wanted to discuss
> > things as a group to see if we could streamline some things, add others,
> > etc.
>
> Well, I for one find the stage field useful.  It reminds me at a glance
> where the issue is at in the workflow (and 'not set' is a valid place
> in the workflow that an issue sometimes stays in for a while for reason
> other than not having been triaged yet).  I suspect that if we discussed
> it as a group (we who make the most changes on the tracker) we could
> improve it, and the interface in general.
>
> By the way, you could talk to people who aren't going to be at the
> summit on #python-dev; I think all the currently tracker-active people
> hang out there on a regular basis.
>
>
Sounds reasonable.


> I'll have to give some thought to what changes/improvements might be
> most useful, now that I've been doing this for a while.


How about everyone who is active on bugs.python.org give it some thought and
we can try to have a discussion at some point in the relatively near future.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/26a09128/attachment-0004.htm>

From ncoghlan at gmail.com  Tue Jan 12 21:58:49 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 06:58:49 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<4B4C749E.4040009@egenix.com>
	<bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
Message-ID: <4B4CE289.6040709@gmail.com>

Brett Cannon wrote:
> I expect there will be at least summary emails on what gets discussed.
> There is also a chance that it will be videotaped.

The Wiki makes for a better summary archive though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Tue Jan 12 22:53:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:53:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
Message-ID: <4B4CEF3D.8000107@v.loewis.de>

>> a) telling people that they have to move to 2.6 first actually
>>   hurts migration, instead of helping, because it implies to them
>>   that they have to drop old versions (e.g. 2.3.) - just because
>>   they had *always* dropped old versions before supporting new ones.
> 
> Is it just an implication, or is it reality?

That's only the implication. However, this was precisely the dialogue
when talking to Django. If you start with "start supporting 2.6", the
immediate response, without listening further, was, "ok, wait until we
drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).

Then explain it to the individual you are talking to, wait for the next
developer of the project step along, and see how he brings up the
very same line of thinking (supporting new versions == dropping support
for old versions).

I think only part of that comes from the maintenance burden. The other
part is that they *want* to drop support for old versions, so that they
can eventually start using new features (e.g. generator expressions).
So they welcome the requirement to support new versions as an excuse
to drop old ones ("it is obvious that you have to drop 2.3 to support
3.2"). However, their users then won't let them drop old versions.


> 
>> b) IMO, people also don't gain much by first migrating to 2.6.
>>   In principle, it gives them the opportunity to get py3k warnings.
>>   However, I haven't heard a single positive report where these
>>   warnings have actually helped people in porting. Yours is the
>>   first report saying that you followed the official guideline,
>>   but you didn't state whether doing so actually helped (or whether
>>   you just ported to 2.6 because the guideline told you to).
> 
> Python 2.6 has other useful features, which I want to take advantage of

I think you are a minority with that, being able to actually use the 2.6
features already. Many projects can't, as they have to support at least
2.4 still (so the with statement is right out).

>> Inherently, 2.8 can't improve on that.
> 
> I'm not so sure.  Yes, as a package maintainer there are older versions to
> think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
> released, what will be the minimum version of Python provided by most OS
> vendors (where the majority of Python users probably get their 'python')?

"Current" Linux distributions will have 2.6 then. "Old" installations
will have 2.4.

> I
> guess some people will have to support everything from Python 2.3 to 2.8 but
> you're talking supporting something like a spread of 7 years of Python
> versions.  What other platform do you support for 7 years?

I think 2.3 will really be gone by the time 2.8 might get released. Even
with 2.7, you'd end up with a span of seven years, though.

Python had been supporting Windows 95 for more than 7 years (I think
rather 9 or 10), likewise Windows 3.1 before that. Python 2.7 will
likely still support Windows 2000, which then will be ten years old.

Solaris support will probably go back to Solaris 2.6, which will be
13 years old when Python 2.7 gets released.

It's only the Linux (and OS X) releases that move so quickly.

Regards,
Martin


From martin at v.loewis.de  Tue Jan 12 22:56:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:56:55 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
Message-ID: <4B4CF027.4080007@v.loewis.de>

> Maybe not, but the Distribute feature is there because IMO the
> distutils feature by itself isn't particularily useful. You need to
> write your own distutils extensions, in practice, and they are not
> trivial.

I wouldn't say that. My Django port works with bare distutils (as
does Django itself), and it works fine.

That distribute had to redo it is only because setuptools *replaces*
the build_py command, as does the 2to3 support in distutils. So only
if you have a different build_py already, you can't use what is in
distutils.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Tue Jan 12 23:02:34 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:02:34 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CEF3D.8000107@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>
	<4B4CEF3D.8000107@v.loewis.de>
Message-ID: <4B4CF17A.1000503@voidspace.org.uk>

On 12/01/2010 21:53, "Martin v. L?wis" wrote:
>>> a) telling people that they have to move to 2.6 first actually
>>>    hurts migration, instead of helping, because it implies to them
>>>    that they have to drop old versions (e.g. 2.3.) - just because
>>>    they had *always* dropped old versions before supporting new ones.
>>>        
>> Is it just an implication, or is it reality?
>>      
> That's only the implication. However, this was precisely the dialogue
> when talking to Django. If you start with "start supporting 2.6", the
> immediate response, without listening further, was, "ok, wait until we
> drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).
>
> Then explain it to the individual you are talking to, wait for the next
> developer of the project step along, and see how he brings up the
> very same line of thinking (supporting new versions == dropping support
> for old versions).
>
> I think only part of that comes from the maintenance burden. The other
> part is that they *want* to drop support for old versions, so that they
> can eventually start using new features (e.g. generator expressions).
> So they welcome the requirement to support new versions as an excuse
> to drop old ones ("it is obvious that you have to drop 2.3 to support
> 3.2"). However, their users then won't let them drop old versions.
>
>
>    

I agree with Martin that the *perception* is that to use Python 2.6 to 
help you port to Python 3 you have to be willing to drop support for 
earlier versions of Python.

>>      
>>> b) IMO, people also don't gain much by first migrating to 2.6.
>>>    In principle, it gives them the opportunity to get py3k warnings.
>>>    However, I haven't heard a single positive report where these
>>>    warnings have actually helped people in porting. Yours is the
>>>    first report saying that you followed the official guideline,
>>>    but you didn't state whether doing so actually helped (or whether
>>>    you just ported to 2.6 because the guideline told you to).
>>>        
>> Python 2.6 has other useful features, which I want to take advantage of
>>      
> I think you are a minority with that, being able to actually use the 2.6
> features already. Many projects can't, as they have to support at least
> 2.4 still (so the with statement is right out).
>
>    
Well, the IronPython community has almost completely moved over to 
IronPython 2.6. :-)

We tend to ship our Python runtime with our applications though. As it 
happens I'm now working with CPython on the server (Python 2.5) and 
IronPython in the browser where we are using 2.6. The new property 
feature is nice, as is having with without a __future__ import.

All the best,


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From regebro at gmail.com  Tue Jan 12 23:03:41 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 23:03:41 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF027.4080007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
	<4B4CF027.4080007@v.loewis.de>
Message-ID: <319e029f1001121403x14ef7ec8x729fb80fa67feaef@mail.gmail.com>

On Tue, Jan 12, 2010 at 22:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Maybe not, but the Distribute feature is there because IMO the
>> distutils feature by itself isn't particularily useful. You need to
>> write your own distutils extensions, in practice, and they are not
>> trivial.
>
> I wouldn't say that. My Django port works with bare distutils (as
> does Django itself), and it works fine.
>
> That distribute had to redo it is only because setuptools *replaces*
> the build_py command, as does the 2to3 support in distutils. So only
> if you have a different build_py already, you can't use what is in
> distutils.

Yeah, you are right, I misremembered. The actual additional feature is
the support for the test command. Testing under Python 2 and 3 without
it is annoying.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan 12 23:04:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:04:37 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <4B4CF1F5.4050403@v.loewis.de>

> [...]
>> I've done a fair bit of 3.x porting, and I'm firmly convinced that
>> 2.x can do nothing:
> [...]
>> Inherently, 2.8 can't improve on that.
> 
> I agree that there are limitations like the ones you've listed, but I
> disagree with your conclusion.  Maybe you assume that it's just as hard
> to move to 2.8 (using the py3k backports not available in say 2.5) as it
> is to 3.x?

Not at all, no. I'd rather expect that code that runs on 2.7 will run
on 2.8 unmodified.

> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. 

How so? If they use anything that is new in 2.8, they *will* need to
drop support for anything before it, no???

> I think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
that help Twisted in moving to 3.2?

> Then, when all of your dependencies (or viable alternatives to those
> dependencies) are available for 3.x, you'll have an easier transition if
> you can start from a 2.x with fewer differences in features.

But you won't *have* fewer differences. Just because your code runs
on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
for that). This doesn't get you any closer - you can't use any of
the 2.8 features as long as you have to support older versions of
Python.

> Fundamentally the more 2.x can converge on 3.x, the easier it will be
> for users to make the leap, because it will be a smaller leap.

No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
likely won't.

> The
> longer the 2.x series lives, the more these newer 2.x versions like 2.7
> and maybe even 2.8 will be available on common platforms for people to
> depend upon as minimum versions, which means that as time goes by they
> can depend on a version that's closer to 3.x.

No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
released in 2010. Then, in 2015, it may be that everybody has migrated
to 2.7, which then is a common platform.

If you release 2.8 in 2012, then, in 2015, people will be split between
2.7 and 2.8, and so there won't be a common platform before 2017.

So stopping 2.x development *earlier* will also give us a common
platform earlier.

Regareds,
Martin


From martin at v.loewis.de  Tue Jan 12 23:07:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:07:02 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <4B4CF286.5040002@v.loewis.de>

> I think it would be interesting to see how people are using the tracker,
> or how they want to be using it. For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all. Would a properly set stage field save issues from
> falling into a black hole?

I personally don't think this would be the case.

What would help is people who actually *work* on the issues, rather than
merely reading them. However, this being open source, there is no way to
force it (unless you create an incentive, such as the five-for-one
deal).

Regards,
Martin


From python at mrabarnett.plus.com  Tue Jan 12 23:10:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 12 Jan 2010 22:10:28 +0000
Subject: [Python-Dev] regex module
Message-ID: <4B4CF354.2050603@mrabarnett.plus.com>

Hi all,

I'm back on the regex module after doing other things and I'd like your
opinion on a number of matters:

Firstly, the current re module has a bug whereby it doesn't split on
zero-width matches. The BDFL has said that this behaviour should be
retained by default in case any existing software depends on it. My
question is: should my regex module still do this for Python 3?
Speaking personally, I'd like it to behave correctly, and Python 3 is
the version where backwards-compatibility is allowed to be broken.

Secondly, Python 2 is reaching the end of the line and Python 3 is the
future. Should I still release a version that works with Python 2? I'm
thinking that it could be confusing if new regex module did zero-width
splits correctly in Python 3 but not in Python 2. And also, should I
release it only for Python 3 as a 'carrot'?

Finally, the module allows some extra backslash escapes, eg \g<name>, in
the pattern. Should it treat ill-formed escapes, eg \g, as it would have
treated them in the re module?

Thanks


From michael at voidspace.org.uk  Tue Jan 12 23:46:49 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:46:49 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
Message-ID: <4B4CFBD9.1090009@voidspace.org.uk>

I presume the email below is about the Windows binary. Does the AMD64 
release work on intel 64bit and can we make the wording clearer on the 
download page?

The current description is " Windows AMD64 binary".

All the best,

Michael

-------- Original Message --------
Subject: 	Download Page - AMD64
Date: 	Fri, 11 Dec 2009 04:03:16 -0600
From: 	Thomas Brownback <thomas.brownback at gmail.com>
To: 	webmaster at python.org



Should I use the AMD64 version of Python on an Intel 64 chip? I know 
those 64-bit implementations are very similar, but are they similar 
enough that your AMD64 will work on Intel?

If so, perhaps a note should be placed on that page to help avoid 
confusion. Explicit being better than implicit and all.

Thanks for your time,
Thomas

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/80d6b539/attachment-0004.htm>

From andrew at bemusement.org  Tue Jan 12 23:49:56 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Wed, 13 Jan 2010 09:49:56 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100112224956.GC28576@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > But a hypothetical 2.8 would also give people a way to move closer to
> > py3k without giving up on using all their 2.x-only dependencies. 
> 
> How so? If they use anything that is new in 2.8, they *will* need to
> drop support for anything before it, no???
> 
> > I think it's much more likely that libraries like Twisted can support 2.8
> > in the near future than 3.x.
> 
> Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
> that help Twisted in moving to 3.2?

I'm not talking about Twisted moving to 3.x (FWIW, I think the only
movement there so far is some patches for some -3 warnings).  The
situation I'm describing is a project X that:

  (a) has 2.x-only dependencies, and
  (b) would like to be as close as possible to 3.x (because they like
      the new features and/or want to be as ready as possible to jump
      when (a) is fixed).

So just because project X depends on e.g. Twisted, and that Twisted in
turn still supports 2.4, doesn't mean that X cannot move to 2.8, and
doesn't mean it would get no benefit from doing so.

[...]
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.

But this is my point.  I think they would as an intermediate step to
jumping to 3.x (which also requires dropping 2.5, after all!), if for
some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

-Andrew.



From tjreedy at udel.edu  Tue Jan 12 23:51:31 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 17:51:31 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <hiiudf$2uv$1@ger.gmane.org>

On 1/12/2010 5:04 PM, "Martin v. L?wis" wrote:

> But you won't *have* fewer differences. Just because your code runs
> on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
> for that). This doesn't get you any closer - you can't use any of
> the 2.8 features as long as you have to support older versions of
> Python.
>
>> Fundamentally the more 2.x can converge on 3.x, the easier it will be
>> for users to make the leap, because it will be a smaller leap.
>
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.
>
>> The
>> longer the 2.x series lives, the more these newer 2.x versions like 2.7
>> and maybe even 2.8 will be available on common platforms for people to
>> depend upon as minimum versions, which means that as time goes by they
>> can depend on a version that's closer to 3.x.
>
> No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
> released in 2010. Then, in 2015, it may be that everybody has migrated
> to 2.7, which then is a common platform.
>
> If you release 2.8 in 2012, then, in 2015, people will be split between
> 2.7 and 2.8, and so there won't be a common platform before 2017.

Just like people today may need to work with both 2.5 and 2.6, or 
privately backport 2.6 bugfixes to 2.5.

> So stopping 2.x development *earlier* will also give us a common
> platform earlier.

With years of bug fixes and hence high quality.




From tjreedy at udel.edu  Wed Jan 13 00:05:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 18:05:12 -0500
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <hiiv75$5md$1@ger.gmane.org>

On 1/12/2010 5:10 PM, MRAB wrote:
> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.

Are you writing a new module with a new name? If so, do you expect it to 
replace or augment re? (This is the same question as for optparse vs. 
argparse, which I understand to not yet be decided.)
>
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?

2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
2.7 stdlib is already out. A new engine should get some community 
testing before going in the stdlib. Even 3.2 beta is not that far off 
(8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?

> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?

What does re do with analogous cases?

Terry Jan Reedy



From david.lyon at preisshare.net  Wed Jan 13 00:20:54 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 13 Jan 2010 10:20:54 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C4589.70906@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<4B4C4589.70906@gmail.com>
Message-ID: <1333.218.214.45.58.1263338454.squirrel@syd-srv02.ezyreg.com>

> Nick wrote:
>>> This has nothing to do with pushing 3.x, but all with managing
>>> available manpower and still providing quality software.
>>
>> Python 3.x needs more carrots.
>
> As Guido has said a few times, the gains are far greater for *new*
> Python developers than they are for existing ones.

Well both you and Guido are most likely 100% correct. :-)

> They don't have as rich a 3rd party ecosystem
> on Python 3 as they would on Python 2.x at this point in time, but
> unlike existing developers they also have the luxury of cherry-picking
> just those packages that already have Python 3 support.

Most likely. I wouldn't want to say anything to discourage
people from going to python 3. In other languages, I have
much experience of making the jump to 'a whole new world'.

It's unsettling at first, but after that, you suddenly
find the new model has a better turbo than the last and
you're at the next corner faster than you can think. So
it's all good.

But I still maintain Python 3.0 needs more carrots. For example,
if mercurial or any other cool lib gets added to python 3 (and
I can name a few that should go in python 3) then they should
be added to python 3 and not to python 2.x

They would serve as good carrots.

Make fresh the python 3 stdlib and preserve the python 2.x stdlib.

I really think we are somewhat short on resources to do
what Guido has asked about bringing python up to CPAN level
with respect to packages.

We're starting a long way back with horses and swords and
trying to catch a well fed and greased modern machine..

I'm sure we can get a modern package testbot operational
for python.

But I wish those who were complaining about the packaging
problem so much could throw some money at the problem
instead of moaning about it. As is done on other open-source
projects. Some organisation would be beneficial.

Finding funds so that a small group of people could
work on the problem would be a great boost to forward
progress. I just think python packaging needs six
months of that to repair the years and years of neglect
and stagnation. Even if it is only beer and bus fare
money. It just needs a temporary shot of adrenalin in
the arm.. so to speak..

Regards

David











From martin at v.loewis.de  Wed Jan 13 00:28:14 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 00:28:14 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D058E.4030404@v.loewis.de>

> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?

"intel 64bit" is as clear as mud. It could mean the "Intel 64"
architecture, or it could mean the "IA-64" architecture, both
are 64-bit architectures with processors manufactured by Intel.
The former is officially called AMD64 (not by Intel, but
officially), and our AMD64 binaries run on such processors.
The latter is also called "Itanium", and we stopped producing
binaries for that architecture a while ago; the AMD64 binaries
will *not* run on it.

The wording could probably be changed to match the reader's
expectation more, most likely, it would also get more incorrect
in the process. To make it correct and explicit, an entire
paragraph educating users about 64-bit architectures and that
there are many of them and that they are mutually incompatible
might be required.

Most likely, Thomas' processor implements the AMD64 architecture,
even though it is manufactured by Intel. A short sentence that
he would probably understand (given that he used "Intel 64", not
"64bit") is:

"""The binaries for AMD64 will also work on processors that implement
the Intel 64 architecture (formerly EM64T), i.e. the architecture that
Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
They will not work on Intel Itanium Processors (formerly IA-64)."""

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 00:30:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 00:30:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112224956.GC28576@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>	<4B4CF1F5.4050403@v.loewis.de>
	<20100112224956.GC28576@steerpike.home.puzzling.org>
Message-ID: <4B4D0613.1010503@v.loewis.de>

> I'm not talking about Twisted moving to 3.x (FWIW, I think the only
> movement there so far is some patches for some -3 warnings).  The
> situation I'm describing is a project X that:
> 
>   (a) has 2.x-only dependencies, and
>   (b) would like to be as close as possible to 3.x (because they like
>       the new features and/or want to be as ready as possible to jump
>       when (a) is fixed).

This assumes that jumping to 3.x is easier if you are closer to it.
Please trust me that this is not the case.

>> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
>> likely won't.
> 
> But this is my point.  I think they would as an intermediate step to
> jumping to 3.x (which also requires dropping 2.5, after all!), if for
> some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

No, and no. No: it's not an intermediate step, and no, supporting 3.x
does *not* require dropping 2.5.

Regards,
Martin



From fuzzyman at voidspace.org.uk  Wed Jan 13 00:40:35 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:40:35 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D058E.4030404@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
Message-ID: <4B4D0873.5070006@voidspace.org.uk>

On 12/01/2010 23:28, "Martin v. L?wis" wrote:
> [snip...]
> """The binaries for AMD64 will also work on processors that implement
> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
> They will not work on Intel Itanium Processors (formerly IA-64)."""
>
>    

To those not intimately aware of the history and present of 64 bit 
architecture this sentence would be a great addition - thanks. I'm 
adding it now as a footnote.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From lists at cheimes.de  Wed Jan 13 00:41:04 2010
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 13 Jan 2010 00:41:04 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D0890.2030505@cheimes.de>

Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64 
> release work on intel 64bit and can we make the wording clearer on the 
> download page?
> 
> The current description is " Windows AMD64 binary".

The installer works on all AMD64 compatible Intel CPUs. *scnr*

As you most likely know all modern Intel 64bit CPUs are based on AMD's
x86-64 design. Only the Itanium family is based on the other Intel 64bit
design: IA-64. The name AMD64 was chosen because most Linux and BSD
distributions call it so. The name 'AMD64' has caused confusion in the
past because users thought that the installer works only on AMD CPUs.

How about:

 * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
X86-64 binary -- does not include source)

instead of:

 * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
not include source)

?

Christia


From fuzzyman at voidspace.org.uk  Wed Jan 13 00:55:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:55:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0873.5070006@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
	<4B4D0873.5070006@voidspace.org.uk>
Message-ID: <4B4D0BD4.5050109@voidspace.org.uk>

On 12/01/2010 23:40, Michael Foord wrote:
> On 12/01/2010 23:28, "Martin v. L?wis" wrote:
>> [snip...]
>> """The binaries for AMD64 will also work on processors that implement
>> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
>> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
>> They will not work on Intel Itanium Processors (formerly IA-64)."""
>>
>
> To those not intimately aware of the history and present of 64 bit 
> architecture this sentence would be a great addition - thanks. I'm 
> adding it now as a footnote.
>

I can add it now to the main download page and existing release pages - 
but are there changes needed to any scripts to change pages created for 
*new* releases?

All the best,

Michael Foord

> All the best,
>
> Michael Foord
>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From python at mrabarnett.plus.com  Wed Jan 13 01:22:01 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 00:22:01 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <hiiv75$5md$1@ger.gmane.org>
References: <4B4CF354.2050603@mrabarnett.plus.com> <hiiv75$5md$1@ger.gmane.org>
Message-ID: <4B4D1229.70708@mrabarnett.plus.com>

Terry Reedy wrote:
> On 1/12/2010 5:10 PM, MRAB wrote:
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
> 
> Are you writing a new module with a new name? If so, do you expect it to 
> replace or augment re? (This is the same question as for optparse vs. 
> argparse, which I understand to not yet be decided.)
 >
It's a module called 'regex'. It can be used in place of 're' by using
"import regex as re", except for differences such as "\g<name>" being a
legal group reference in pattern strings.
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
> 
> 2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
> 2.7 stdlib is already out. A new engine should get some community 
> testing before going in the stdlib. Even 3.2 beta is not that far off 
> (8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?
> 
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
> 
> What does re do with analogous cases?
> 
The 're' module treats r"\g" as "g"; both 're' and 'regex' treat, say, 
r"\q" as "q". The closest analogue to what I'm asking about is that re
treats the ill-formed repeat r"x{1," as a literal, which sort of
suggests that r"\g" should be treated as "g", but r"\g<name>" is now a
group reference (re would treat that as "g<name>". Does that sound
reasonable?


From fuzzyman at voidspace.org.uk  Wed Jan 13 01:50:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 00:50:39 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D18DF.6070502@voidspace.org.uk>

On 12/01/2010 23:41, Christian Heimes wrote:
> Michael Foord wrote:
>    
>> I presume the email below is about the Windows binary. Does the AMD64
>> release work on intel 64bit and can we make the wording clearer on the
>> download page?
>>
>> The current description is " Windows AMD64 binary".
>>      
> The installer works on all AMD64 compatible Intel CPUs. *scnr*
>
> As you most likely know all modern Intel 64bit CPUs are based on AMD's
> x86-64 design. Only the Itanium family is based on the other Intel 64bit
> design: IA-64. The name AMD64 was chosen because most Linux and BSD
> distributions call it so. The name 'AMD64' has caused confusion in the
> past because users thought that the installer works only on AMD CPUs.
>
> How about:
>
>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
>
> instead of:
>
>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)
>    
Right - I've made that change for the Python 2.6, 2.7, 3.0 and 3.1 
download pages with a footnote. Prior to that we were offering ia64 
*and* amd64 releases so I've left those unchanged.

Thanks,

Michael

> ?
>
> Christia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From exarkun at twistedmatrix.com  Wed Jan 13 02:40:03 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Jan 2010 01:40:03 -0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100113014003.1898.1305834328.divmod.xquotient.219@localhost.localdomain>

On 12 Jan, 10:04 pm, martin at v.loewis.de wrote:
>>[...]
>>>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>>>2.x can do nothing:
>>[...]
>>>Inherently, 2.8 can't improve on that.
>>
>>I agree that there are limitations like the ones you've listed, but I
>>disagree with your conclusion.  Maybe you assume that it's just as 
>>hard
>>to move to 2.8 (using the py3k backports not available in say 2.5) as 
>>it
>>is to 3.x?
>
>Not at all, no. I'd rather expect that code that runs on 2.7 will run
>on 2.8 unmodified.
>>But a hypothetical 2.8 would also give people a way to move closer to
>>py3k without giving up on using all their 2.x-only dependencies.
>
>How so? If they use anything that is new in 2.8, they *will* need to
>drop support for anything before it, no???
>>I think it's much more likely that libraries like Twisted can support 
>>2.8
>>in the near future than 3.x.
>
>Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
>that help Twisted in moving to 3.2?

I'm not reading this thread carefully enough to make any arguments on 
either side, but I can contribute a fact.

Twisted very likely does not support 2.8 today.  I base this on the fact 
that Twisted does not support 2.7 today, and I expect 2.8 will be more 
like 2.7 than it will be like 2.3 - 2.6 (which Twisted does support).

When I say "support" here, I mean "all of the Twisted unit tests pass on 
it".

Jean-Paul


From tseaver at palladion.com  Wed Jan 13 03:57:46 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 12 Jan 2010 21:57:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <4B4D36AA.7020607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Karen Tracey wrote:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>>
> Well yay.  Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.  That's
> down from 104 failures and 40 errors with 2.7 alpha 1.
> 
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Just to add another "success" data point:  Zope2's trunk, as well as the
2.12 release, passes all tests (2535 on the trunk) and brings up the
appserver just fine under 2.7a2.

There is an obnoxious deprecation warning out of the distutils:

  DeprecationWarning: 'compiler' specifies the compiler type in
  build_ext. If you want to get the compiler object itself, use
  'compiler_obj'

which is likely a simple one-line fix, if I only knew what the hell it
was whining about. ;)  The warning is extra obnoxious because it doesn't
tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
argument).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktNNqQACgkQ+gerLs4ltQ7d5gCcCGKVjyOlxnrAln0UnRibS7kd
lNIAoIs1RlSGMtJWaY11BqptfDmQvR87
=mIOO
-----END PGP SIGNATURE-----



From python at mrabarnett.plus.com  Wed Jan 13 04:09:34 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 03:09:34 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <4B4D396E.4050500@mrabarnett.plus.com>

MRAB wrote:
> Hi all,
> 
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
> 
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
> 
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
> 
> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
> 
I've just noticed something odd about the re module: the sub() method
doesn't take 'pos' or 'endpos' arguments. search() does; match() does;
findall() does(); finditer() does; but sub() doesn't. Maybe there has
never been a demand for it. (Nor split(), for that matter.)


From brett at python.org  Wed Jan 13 04:58:11 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 19:58:11 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>

On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:

> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
>
>
If it is a separate module under a different name it can do the proper
thing. People will just need to be aware of the difference when they import
the module.


> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
>
>
That's totally up to you. There is practically no chance of it getting into
the 2.x under the stdlib at this point since 2.7b1 is coming up and this
module has not been out in the wild for a year (to my knowledge).  If you
want to support 2.x that's fine and I am sure users would appreciate it, but
it isn't necessary to get into the Python 3 stdlib.


> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
>
>
If you want to minimize the differences then it should probably match. As I
said, since it is a different name to import under it can deviate where
reasonable, just make sure to clearly document the deviations.

-Brett



> Thanks
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/f1f73d56/attachment-0004.htm>

From martin at v.loewis.de  Wed Jan 13 07:11:49 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 07:11:49 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D6425.3060307@v.loewis.de>

> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
> 
> instead of:
> 
>  * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)

-1. AMD doesn't want us to use the term x86-64 anymore, but wants us
to use AMD64 instead. I think we should comply - they invented the
architecture, so they have the right to give it a name. Neither
Microsoft nor Intel have such a right.

Regards,
Martin


From sridharr at activestate.com  Wed Jan 13 07:47:38 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 12 Jan 2010 22:47:38 -0800
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D6C8A.7070307@activestate.com>

On 1/12/2010 2:46 PM, Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?
>
> The current description is " Windows AMD64 binary".

FWIW, we simply use (64-bit, x64).

   Platform	          Download
   ==============================================
   Windows (x86)	          Windows Installer (MSI)
   Windows (64-bit, x64)	  Windows Installer (MSI)

https://www.activestate.com/activepython/downloads/

-srid


From solipsis at pitrou.net  Wed Jan 13 08:08:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 13 Jan 2010 07:08:55 +0000 (UTC)
Subject: [Python-Dev] Fwd: Download Page - AMD64
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <loom.20100113T080717-468@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)

+1. I don't care about trademarks or official names, we should call it whatever
is obvious for our users.
As for Itanium, it is practically dead, I think there is little chance of people
mixing it up with x86-64.

cheers

Antoine.




From michael at voidspace.org.uk  Wed Jan 13 10:32:00 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 09:32:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <4B4D9310.6060907@voidspace.org.uk>

On 13/01/2010 06:11, "Martin v. L?wis" wrote:
>> How about:
>>
>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
>>
>> instead of:
>>
>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>> not include source)
>>      
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.
>    

I think we should use whatever is most informative and least confusing 
to our users - we owe our allegiance to them and not to a processor vendor.

Michael


> Regards,
> Martin
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ncoghlan at gmail.com  Wed Jan 13 11:30:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:30:50 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF17A.1000503@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk>
Message-ID: <4B4DA0DA.7070007@gmail.com>

Michael Foord wrote:
> I agree with Martin that the *perception* is that to use Python 2.6 to
> help you port to Python 3 you have to be willing to drop support for
> earlier versions of Python.

Note that increased 3.x compatibility in the most recent 2.x release
will always help in two scenarios:

1. New projects that want to use 2.x only libraries but want to be ready
for the Py3k transition in their own code (e.g. new 2.7 features like
set literals, dict and set comprehensions and the view* dict methods can
be translated far more cleanly by 2to3 than the closest comparable 2.6
code).

2. Similarly new projects that use a 3.x trunk can be more readily
translated to a 2.7 version with 3to2, whereas some constructs may be
difficult to recreate in earlier Python versions. I would expect this to
be significantly less common then the first scenario though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:38:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:38:31 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <loom.20100113T080717-468@post.gmane.org>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<loom.20100113T080717-468@post.gmane.org>
Message-ID: <4B4DA2A7.2080408@gmail.com>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> How about:
>>
>>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
> 
> +1. I don't care about trademarks or official names, we should call it whatever
> is obvious for our users.

Until this discussion, I didn't even know the architecture had any name
other than x86-64.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:39:40 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:39:40 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4D36AA.7020607@palladion.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com>
Message-ID: <4B4DA2EC.3050908@gmail.com>

Tres Seaver wrote:
> There is an obnoxious deprecation warning out of the distutils:
> 
>   DeprecationWarning: 'compiler' specifies the compiler type in
>   build_ext. If you want to get the compiler object itself, use
>   'compiler_obj'
> 
> which is likely a simple one-line fix, if I only knew what the hell it
> was whining about. ;)  The warning is extra obnoxious because it doesn't
> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
> argument).

Could you kick a tracker issues in Tarek's direction for that one?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From vinay_sajip at yahoo.co.uk  Wed Jan 13 13:24:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 13 Jan 2010 12:24:09 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <loom.20100113T131816-515@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> If there is something missing from the stdlib discussion that you think should
be brought up at the summit let me know. And if there is something here you want
to discuss before the summit let me know and I can start a separate thread on it.

I'm sorry I won't be able to come to the language summit, but I would like if
possible to expedite a pronouncement on PEP 391 (configuring logging using
dictionaries). I believe I addressed all the comments made on the discussion
threads mentioned in the PEP and so I'm not sure what more I need to do to get a
pronouncement. I guess the stdlib slot gives an opportunity for people to air
their views and so I'd be grateful if you added it to the agenda.

Thanks & regards,

Vinay Sajip



From murman at gmail.com  Wed Jan 13 14:35:51 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 07:35:51 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>

On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.

I see and agree with the motivation behind your point, but it's just
as reasonable to mimic the name Microsoft uses: the release is for
Windows rather than the processor. On MSDN Microsoft uses
parentheticals x86, ia64 and x64; this would suggest a name like
Python 2.6.4 installer for Windows (x64). Unfortunately this usage
doesn't seem to be reflected in consumer-oriented product pages, so
I'm uncertain how clear it is for those downloading Python.

-- 
Michael Urman


From ncoghlan at gmail.com  Wed Jan 13 15:04:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 00:04:28 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
Message-ID: <4B4DD2EC.3030709@gmail.com>

Michael Urman wrote:
> On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
> 
> I see and agree with the motivation behind your point, but it's just
> as reasonable to mimic the name Microsoft uses: the release is for
> Windows rather than the processor. On MSDN Microsoft uses
> parentheticals x86, ia64 and x64; this would suggest a name like
> Python 2.6.4 installer for Windows (x64). Unfortunately this usage
> doesn't seem to be reflected in consumer-oriented product pages, so
> I'm uncertain how clear it is for those downloading Python.

As Windows doesn't run on non-x86 architectures, their naming is
generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Linux, on the other hand, can run on multiple 64 bit architectures, so
being more specific and using the official AMD64 nomenclature makes sense.

In this case, making it clear that the 64-bit Windows binaries work for
both Intel and AMD chips is more important than using only the official
architecture name.

Cheers,
Nick.

P.S. This discussion made me realise that out of habit I had dropped the
32-bit version of Python on my new 64-bit Windows box...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tseaver at palladion.com  Wed Jan 13 17:49:55 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 13 Jan 2010 11:49:55 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA2EC.3050908@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com> <4B4DA2EC.3050908@gmail.com>
Message-ID: <4B4DF9B3.4030403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> There is an obnoxious deprecation warning out of the distutils:
>>
>>   DeprecationWarning: 'compiler' specifies the compiler type in
>>   build_ext. If you want to get the compiler object itself, use
>>   'compiler_obj'
>>
>> which is likely a simple one-line fix, if I only knew what the hell it
>> was whining about. ;)  The warning is extra obnoxious because it doesn't
>> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
>> argument).
>
> Could you kick a tracker issues in Tarek's direction for that one?

Done:

  http://bugs.python.org/issue7694


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktN+bMACgkQ+gerLs4ltQ6s4gCdENhtVQWzoH449BCDz8SNx/4s
DEoAn19LMcMs3b6ctdNF8K2hYd6d+BSZ
=TWit
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Jan 13 18:24:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jan 2010 12:24:42 -0500
Subject: [Python-Dev] PYTHON3PATH
Message-ID: <20100113172442.4A7771FA679@kimball.webabinitio.net>

Please review issue 2375 [1], which is an enhancement request to add a
PYTHON3PATH environment variable.  Because we have elected to have both
a python and a python3 command, I think this is an issue worth thinking
about carefully to make sure we are serving the Python user community
and easing the transition to python3.  It could be that "use virtualenv"
is the best answer, but I feel we should think about it carefully to
make sure that is really true.

[1] http://bugs.python.org/issue2375

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From guido at python.org  Wed Jan 13 18:31:39 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:31:39 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>

Somehow the bug site doesn't load for me right now, but I'm -1 on
this. There are maybe a dozen PYTHON... variables -- we really
shouldn't try to add PYTHON3 variants for all of them.

Specifically for PYTHONPATH, I feel that its use is always a
short-time or localized hack, not something you set in your .profile
nd forget about it (forgetting about it inparticular often leads to
unnecessary debugging pain later). The better approaches are based on
site-packages, wrapper scripts, virtualenv, etc.

--Guido

On Wed, Jan 13, 2010 at 9:24 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable. ?Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3. ?It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375
>
> --
> R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From dirkjan at ochtman.nl  Wed Jan 13 18:38:26 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 13 Jan 2010 18:38:26 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <ea2499da1001130938v68827914yc41476c75b5f3c62@mail.gmail.com>

On Sat, Jan 9, 2010 at 18:29, Benjamin Peterson <benjamin at python.org> wrote:
> Please consider trying Python 2.7 with your code and reporting any bugs you may

I ran the Mercurial test suite on current trunk, and it worked
alright. There was a small issue with the fact that mimetypes got
fooled by the lack of ImportError from _winreg (which I solved by
blacklisting _winreg in demandimport), and one test fails likely
because of a change in MIME handling. Will look into that. So the
state of trunk looks rather solid from where I sit.

Cheers,

Dirkjan


From ralf at brainbot.com  Wed Jan 13 18:40:04 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:40:04 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net> (R. David
	Murray's message of "Wed, 13 Jan 2010 12:24:42 -0500")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <87r5ptdhu3.fsf@muni.brainbot.com>

"R. David Murray" <rdmurray at bitdance.com> writes:

> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable.  Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3.  It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375

The first thing I got while trying to run a python3 prompt few days ago,
was an error. python3 tried to read my $PYTHONSTARTUP file, which used
print statements. people will have to run both python 2 and python 3
code at the same time. Using different environment variables will make
this easier.

- Ralf


From ralf at brainbot.com  Wed Jan 13 18:55:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:55:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
	(Guido van Rossum's message of "Wed, 13 Jan 2010 09:31:39 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
Message-ID: <87k4vldh4c.fsf@muni.brainbot.com>

Guido van Rossum <guido at python.org> writes:

> Somehow the bug site doesn't load for me right now, but I'm -1 on
> this. There are maybe a dozen PYTHON... variables -- we really
> shouldn't try to add PYTHON3 variants for all of them.
>
> Specifically for PYTHONPATH, I feel that its use is always a
> short-time or localized hack, not something you set in your .profile
> nd forget about it (forgetting about it inparticular often leads to
> unnecessary debugging pain later). The better approaches are based on
> site-packages, wrapper scripts, virtualenv, etc.

then at least remove them completely from python3, so one can use them
for his python 2 installation. Otherwise it will stump on some innocent
python 3 program (and cause unnecessary debugging pain).



From steven.bethard at gmail.com  Wed Jan 13 18:57:29 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 13 Jan 2010 09:57:29 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:40 AM, Ralf Schmitt <ralf at brainbot.com> wrote:
> "R. David Murray" <rdmurray at bitdance.com> writes:
>
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable. ?Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3. ?It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>>
>> [1] http://bugs.python.org/issue2375
>
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

How complicated is your PYTHONSTARTUP file? My suspicion is that you
could easily write it to work for both Python 2.X and 3.X.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From ssteinerx at gmail.com  Wed Jan 13 18:57:42 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 13 Jan 2010 12:57:42 -0500
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>


On Jan 13, 2010, at 12:40 PM, Ralf Schmitt wrote:

> "R. David Murray" <rdmurray at bitdance.com> writes:
> 
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable.  Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3.  It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>> 
>> [1] http://bugs.python.org/issue2375
> 
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

Or, how about just removing the antiquated use of environment variables altogether from Python 3 and avoid the issue completely.  

Python 2 will continue to run as it has, and better solutions will emerge for Python 3 development.

Beats the heck out of making a whole duplicate set for PYTHON3BLAHBLAH, yuck!

S



From guido at python.org  Wed Jan 13 18:58:04 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:58:04 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
	<bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
Message-ID: <ca471dc21001130958k6edabaacof256b5e811c75ea6@mail.gmail.com>

Memories of days past... Python had several regular expression
implementations before, one of which was called "regex".

But I would rather not have a new module -- I would much rather have a
flag specifying the new (backwards incompatible) syntax/semantics. The
flag would have a long name (e.g. re.NEW_SYNTAX), a short name (e.g.
re.N) and an inline syntax, "(?n)...".

--Guido

On Tue, Jan 12, 2010 at 7:58 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
>>
>
> If it is a separate module under a different name it can do the proper
> thing. People will just need to be aware of the difference when they import
> the module.
>
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
>>
>
> That's totally up to you. There is practically no chance of it getting into
> the 2.x under the stdlib at this point since 2.7b1 is coming up and this
> module has not been out in the wild for a year (to my knowledge). ?If you
> want to support 2.x that's fine and I am sure users would appreciate it, but
> it isn't necessary to get into the Python 3 stdlib.
>
>>
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
>>
>
> If you want to minimize the differences then it should probably match. As I
> said, since it is a different name to import under it can deviate where
> reasonable, just make sure to clearly document the deviations.
> -Brett
>
>>
>> Thanks
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)


From ralf at brainbot.com  Wed Jan 13 19:03:08 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 19:03:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
	(Steven Bethard's message of "Wed, 13 Jan 2010 09:57:29 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
Message-ID: <87fx69dgrn.fsf@muni.brainbot.com>

Steven Bethard <steven.bethard at gmail.com> writes:

>
> How complicated is your PYTHONSTARTUP file? My suspicion is that you
> could easily write it to work for both Python 2.X and 3.X.

sure. that's exactly what I did. My point is that sharing those
environment variables will cause pain for some people.


From guido at python.org  Wed Jan 13 19:07:58 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:07:58 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
just removing the antiquated use of environment variables altogether
from Python 3 and avoid the issue completely.

-1. They have their use, but more in controlled situations. If you
have "global" env vars that you only want to use with Python 2.x,
write a wrapper for Python 3 that invokes it with -E.

-- 
--Guido van Rossum (python.org/~guido)


From scott+python-dev at scottdial.com  Wed Jan 13 19:14:21 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 13 Jan 2010 13:14:21 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E0D7D.3040806@scottdial.com>

On 1/13/2010 9:04 AM, Nick Coghlan wrote:
> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

That is not correct. There are IA-64 versions of Window Server.

>From [1]:
"Backward compatibility is a key point differentiating Itanium from x86
and x64 architectures."

So to echo what Michael said, the Microsoft nomenclature is "x64"
regardless of yours and Martin's objections to that name. Nobody who
uses Windows would be confused by "x64" since that is *the* Microsoft
naming scheme. And, anyone using IA64 will expect to see "IA64" and not
"x64", and furthermore anyone using IA64 is likely aware of what
processor they are using (being as there are only Windows Server
editions for IA64) and the difference between "IA64" and "x64".

I don't think an end-user facing page is a great place for pedantic and
wordy defenses of defying the Microsoft-blessed naming scheme.

> Linux, on the other hand, can run on multiple 64 bit architectures, so
> being more specific and using the official AMD64 nomenclature makes
> sense.

This has no relevance to the conversation since there are no Linux
binaries being distributed. The conversation on the expectations of
Windows end-users, who are the target of the download links.

[1] http://www.microsoft.com/servers/64bit/itanium/overview.mspx

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From guido at python.org  Wed Jan 13 19:22:33 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:22:33 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <loom.20100113T131816-515@post.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<loom.20100113T131816-515@post.gmane.org>
Message-ID: <ca471dc21001131022w429e4ceal899235bc711ecaca@mail.gmail.com>

On Wed, Jan 13, 2010 at 4:24 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> If there is something missing from the stdlib discussion that you think should
> be brought up at the summit let me know. And if there is something here you want
> to discuss before the summit let me know and I can start a separate thread on it.
>
> I'm sorry I won't be able to come to the language summit, but I would like if
> possible to expedite a pronouncement on PEP 391 (configuring logging using
> dictionaries). I believe I addressed all the comments made on the discussion
> threads mentioned in the PEP and so I'm not sure what more I need to do to get a
> pronouncement. I guess the stdlib slot gives an opportunity for people to air
> their views and so I'd be grateful if you added it to the agenda.

Is the PEP in the stage of consensus yet? If so it may not even be
necessary to discuss it at the summit (though I certainly won't stop
people if they want to).

-- 
--Guido van Rossum (python.org/~guido)


From phd at phd.pp.ru  Wed Jan 13 19:18:50 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 21:18:50 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <20100113181850.GA3837@phd.pp.ru>

On Wed, Jan 13, 2010 at 12:57:42PM -0500, ssteinerX at gmail.com wrote:
> Or, how about just removing the antiquated use of environment variables

   "antiquated"? You are kidding, aren't you?!

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From guido at python.org  Wed Jan 13 19:51:14 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:51:14 -0800
Subject: [Python-Dev] PyCon Keynote
Message-ID: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>

Please mail me topics you'd like to hear me talk about in my keynote
at PyCon this year.

-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Wed Jan 13 20:13:58 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:13:58 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D9310.6060907@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de> <4B4D9310.6060907@voidspace.org.uk>
Message-ID: <4B4E1B76.4010309@v.loewis.de>

>>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>> X86-64 binary -- does not include source)
>>>
>>> instead of:
>>>
>>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>> not include source)
>>>      
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
>>    
> 
> I think we should use whatever is most informative and least confusing
> to our users - we owe our allegiance to them and not to a processor vendor.

And why do you think this is x86-64?

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:33:24 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:33:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA0DA.7070007@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk> <4B4DA0DA.7070007@gmail.com>
Message-ID: <4B4E2004.8050905@v.loewis.de>

> Note that increased 3.x compatibility in the most recent 2.x release
> will always help in two scenarios:
> 
> 1. New projects that want to use 2.x only libraries but want to be ready
> for the Py3k transition in their own code (e.g. new 2.7 features like
> set literals, dict and set comprehensions and the view* dict methods can
> be translated far more cleanly by 2to3 than the closest comparable 2.6
> code).

Of these, I can only see this being the case of the view* methods.

Why would set literals allow for code that more cleanly translates
to 3.x? Suppose you write

 f = set((1,2,3))

in 2.6. That code will work *exactly* the same way in 3.1, no
translation needed at all. Likewise for dict and set comprehensions:
the code is as cleanly translated in the 2.7 form as it is in any
prior form (ie. no modifications).

As for view* methods: there is one important exception where
the 2.6 equivalent is as cleanly translated as a view method:

for key in D:
  action

This is a more modern equivalent for iterating over D.keys(),
so if your code still does the latter, just change it to the
former (requires Python 2.2). I'd claim that this is the dominant
case of traversing a dictionary (probably followed by iterating
over .items()). So while it is true that only view* can be
transformed correctly, I'd claim that this is an infrequent
issue.

> 2. Similarly new projects that use a 3.x trunk can be more readily
> translated to a 2.7 version with 3to2, whereas some constructs may be
> difficult to recreate in earlier Python versions.

That may be true, but is besides the point here: the issue was whether
a 2.8 release might help in migrating to 3.x. People who follow this
approach already have 3.x code. Whether they would rather port to 2.8
only, and get that work with little effort, or whether they would port
to 2.5 and later, and put in more work, I don't know - I guess we would
have to ask these people. I would expect that they prefer if 2.x
died rather sooner than later, because they can then stop supporting
it.

Regards,
Martin



From tjreedy at udel.edu  Wed Jan 13 20:36:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 Jan 2010 14:36:03 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <hil7av$52r$1@ger.gmane.org>

On 1/13/2010 2:13 PM, "Martin v. L?wis" wrote:

>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>
> And why do you think this is x86-64?

I do not believe I have personally seen, or at least noticed, that 
specific term before. Something like

"Release for 64-bit Windows (Win NT, 2000, XP, Vista, 7 <or whatever 
correct list is>) on AMD64-type processors from AMD and Intel"

should be clear enough.




From martin at v.loewis.de  Wed Jan 13 20:40:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 20:40:30 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E21AE.40602@v.loewis.de>

> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Windows actually does - it runs on IA-64 (which is a non-x86 architecture).

However, end users are unlikely to use such hardware, so distinguishing
between 32-bit and 64-bit is typically good enough.

> In this case, making it clear that the 64-bit Windows binaries work for
> both Intel and AMD chips is more important than using only the official
> architecture name.

Well, we did have Itanium binaries at one point, and the "64-bit Windows
binaries" you are referring to most definitely don't run on Itanium,
even though that's an Intel chip.

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:45:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:45:40 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E0D7D.3040806@scottdial.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>	<4B4DD2EC.3030709@gmail.com>
	<4B4E0D7D.3040806@scottdial.com>
Message-ID: <4B4E22E4.4040504@v.loewis.de>

> So to echo what Michael said, the Microsoft nomenclature is "x64"
> regardless of yours and Martin's objections to that name. Nobody who
> uses Windows would be confused by "x64" since that is *the* Microsoft
> naming scheme.

That's actually not entirely true. There are several places in the
APIs where Microsoft either allows or requires to call the architecture
AMD64 (e.g. architecture indication in MSI files).

Regards,
Martin


From regebro at gmail.com  Wed Jan 13 20:50:59 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 20:50:59 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>

On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

What do you need to do in the PYTHONSTARTUP file?
Ten years of Python programming, and I didn't even know it existed. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From phd at phd.pp.ru  Wed Jan 13 21:08:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 23:08:40 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <20100113200840.GC14858@phd.pp.ru>

On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From murman at gmail.com  Wed Jan 13 21:12:11 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 14:12:11 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E22E4.4040504@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com> <4B4E0D7D.3040806@scottdial.com>
	<4B4E22E4.4040504@v.loewis.de>
Message-ID: <dcbbbb411001131212q3ef907c8i67e8c03c96c31e2f@mail.gmail.com>

On Wed, Jan 13, 2010 at 13:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So to echo what Michael said, the Microsoft nomenclature is "x64"
>> regardless of yours and Martin's objections to that name. Nobody who
>> uses Windows would be confused by "x64" since that is *the* Microsoft
>> naming scheme.
>
> That's actually not entirely true. There are several places in the
> APIs where Microsoft either allows or requires to call the architecture
> AMD64 (e.g. architecture indication in MSI files).

I should have clarified I was talking about the names shown on MSDN
subscriptions for downloading installation media of Windows 7 and
Windows Vista. It's pretty clear in that context Microsoft uses x64 to
describe this platform of Windows. But again, it's far from clear that
this is a term they use for non-developers.

-- 
Michael Urman


From regebro at gmail.com  Wed Jan 13 21:27:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 21:27:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113200840.GC14858@phd.pp.ru>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<20100113200840.GC14858@phd.pp.ru>
Message-ID: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>

On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>> What do you need to do in the PYTHONSTARTUP file?
>> Ten years of Python programming, and I didn't even know it existed. :-)
>
> ? See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Cheese and fries! :-)

Anyway, I don't see how this could pose any problems, because any user
advanced enough to do these things will be advanced enough to
understand what the problem is and fix it so it works under Python 3
too.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ralf at brainbot.com  Wed Jan 13 21:52:34 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 20:52:34 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	(Lennart Regebro's message of "Wed, 13 Jan 2010 20:50:59 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <874ompg225.fsf@brainbot.com>

Lennart Regebro <regebro at gmail.com> writes:

> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
>
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

hehe. tab completion:

# -*- mode: python -*- 
# Last changed: 2009-12-23 22:25:15 by ralf

import sys
import os

def initreadline():
    
    try:
        import readline
    except ImportError:
        sys.stdout.write("Module readline not available.\n")
        return
    
    import rlcompleter
    readline.parse_and_bind("tab: complete")
    
    # Use tab for completions
    readline.parse_and_bind('tab: complete')
    # This forces readline to automatically print the above list when tab
    # completion is set to 'complete'.
    readline.parse_and_bind('set show-all-if-ambiguous on')
    # Bindings for incremental searches in the history. These searches
    # use the string typed so far on the command line and search
    # anything in the previous input history containing them.
    readline.parse_and_bind('"\C-r": reverse-search-history')
    readline.parse_and_bind('"\C-s": forward-search-history')

    history = os.path.expanduser("~/.pyhistory") 
    if os.path.exists(history):
        readline.read_history_file(history)
        
    import atexit
    atexit.register(lambda: readline.write_history_file(history))
    
initreadline()
del initreadline


From martin at v.loewis.de  Wed Jan 13 22:05:03 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 22:05:03 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <4B4E357F.4050107@v.loewis.de>

Lennart Regebro wrote:
> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
> 
> What do you need to do in the PYTHONSTARTUP file?

I personally set up readline: set tab completion, and load the history
of commands from the previous session.

Of course, I don't know what Ralf uses it for.

Regards,
Martin


From ncoghlan at gmail.com  Wed Jan 13 22:43:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 07:43:41 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
Message-ID: <4B4E3E8D.2010407@gmail.com>

Guido van Rossum wrote:
> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
> just removing the antiquated use of environment variables altogether
> from Python 3 and avoid the issue completely.
> 
> -1. They have their use, but more in controlled situations. If you
> have "global" env vars that you only want to use with Python 2.x,
> write a wrapper for Python 3 that invokes it with -E.

Perhaps a case can be made for Python 3 to assume -E by default, with a
-e option to enable reading of the environment variables?

That way naive users could run Python3 without tripping over existing
Python2 environment variables, while other tools could readily set up a
different environment before launching Python3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From guido at python.org  Wed Jan 13 23:45:57 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 14:45:57 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com> 
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com> 
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <ca471dc21001131445g639c6867ude83f03a77eb72b9@mail.gmail.com>

On Wed, Jan 13, 2010 at 1:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
>
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
>
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users using environment variables? That's a recipe for disaster
in any version! :-)

-- 
--Guido van Rossum (python.org/~guido)


From jared.grubb at gmail.com  Wed Jan 13 23:56:37 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Wed, 13 Jan 2010 14:56:37 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <13F6C43A-D62A-4A9F-AF7A-9BDAC94F7D72@gmail.com>

On 13 Jan 2010, at 13:43, Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>> 
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

I raised a question about these PYTHON3* variables once before in a discussion about shebang lines:
http://www.mailinglistarchive.com/python-dev at python.org/msg29500.html

I'm not advocating them, but just wanted to make sure to bring up the shebang use case.

Jared

From skip at pobox.com  Wed Jan 13 23:24:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 16:24:13 -0600
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <19278.18445.428716.321365@montanaro.dyndns.org>


    Lennart> What do you need to do in the PYTHONSTARTUP file?

Just reading off stuff from my own personal startup file...  I use it for
stuff I want available during interactive sessions:

    1. Enable true division.

    2. Conditionally define "help" from back in the days when there was no
       help builtin function.

    3. Auto-save session (readline) history so I can easily recall commands
       across sessions.

    4. Add other fake builtins ("like") or override behavior of some (like
       "dir") making them handier for interactive use.

    5. autoload modules/symbols (pokes around in common modules from
       sys.excepthook function).

Oh, and I've had no particular trouble keeping it working in Python 1, 2 or
3.

Skip




From fuzzyman at voidspace.org.uk  Thu Jan 14 01:20:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 00:20:21 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <4B4E6345.5070202@voidspace.org.uk>

On 13/01/2010 19:13, "Martin v. L?wis" wrote:
>>>>    * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>>> X86-64 binary -- does not include source)
>>>>
>>>> instead of:
>>>>
>>>>    * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>>> not include source)
>>>>
>>>>          
>>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>>> to use AMD64 instead. I think we should comply - they invented the
>>> architecture, so they have the right to give it a name. Neither
>>> Microsoft nor Intel have such a right.
>>>
>>>        
>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>>      
> And why do you think this is x86-64?
>    

Well anecdotal everyone I have *every* talked to about 64bit processors 
has referred to having a 64bit processor (x86 is a given) and not an 
AMD64 architecture processor.

Linus Torvalds addressed this specific issue for Linux and came down on 
the side of "x86-64": http://kerneltrap.org/node/2466
Look up AMD64 on Wikipedia and it redirects you to the X86-64 page.
Information website setup by AMD and partners about the AMD64 
architecture: http://www.x86-64.org/about.html
In the AMD website they refer to "x86-64 Assembly": 
http://www.x86-64.org/documentation/assembly.html

Microsoft seem to draw a distinction between x64 (which would also be 
acceptable) and Itanium based systems. Very rarely do they refer to AMD64:

* http://www.microsoft.com/servers/64bit/compare.mspx
* http://www.microsoft.com/servers/64bit/x64/overview.mspx
* http://www.microsoft.com/servers/64bit/overview.mspx

Using a vendor specific name automatically begs the question as to 
whether the installer works on processors from other vendors, as we saw 
in the specific enquiry from the user that triggered this debate.

Referring to the AMD 64 build as x86-64, with a footnote explaining 
which architectures this specifically means is unlikely to confuse 
people. It is *definitely* better than just saying AMD64.

All the best,

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From skip at pobox.com  Thu Jan 14 02:50:55 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 19:50:55 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
Message-ID: <19278.30847.649228.115514@montanaro.dyndns.org>


Just out of curiosity, is the static build stuff (use the old Modules/Setup
file to build modules) exercised at all any more?

Skip



From benjamin at python.org  Thu Jan 14 03:22:03 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 13 Jan 2010 20:22:03 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
In-Reply-To: <19278.30847.649228.115514@montanaro.dyndns.org>
References: <19278.30847.649228.115514@montanaro.dyndns.org>
Message-ID: <1afaf6161001131822r151bd202x35653ba44ee0235d@mail.gmail.com>

2010/1/13  <skip at pobox.com>:
>
> Just out of curiosity, is the static build stuff (use the old Modules/Setup
> file to build modules) exercised at all any more?

Exercised as in used in the build process? Yes.


-- 
Regards,
Benjamin


From vinay_sajip at yahoo.co.uk  Thu Jan 14 10:23:47 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 09:23:47 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
Message-ID: <954450.26617.qm@web25804.mail.ukl.yahoo.com>

In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:

  http://www.python.org/dev/peps/pep-0391/

In November 2009 I posted to this list that the PEP was ready for review.

I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.

So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.

Thanks & regards,

Vinay Sajip



      



From ziade.tarek at gmail.com  Thu Jan 14 10:30:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 Jan 2010 10:30:15 +0100
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <94bdd2611001140130u6154e893jfd37db32a25be66a@mail.gmail.com>

On Thu, Jan 14, 2010 at 10:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[..]
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would
> be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check
> in the changes unless there are objections to doing so. All those who think that logging is currently
> hard to configure will benefit from these changes, so here's the opportunity to pipe up and
> improve things.


FWIW, I am +1. Thanks for your work

Tarek


From hodgestar+pythondev at gmail.com  Thu Jan 14 10:39:41 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 14 Jan 2010 11:39:41 +0200
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <fb73205e1001140139n4b920871t6bfc6b91c469264f@mail.gmail.com>

On Thu, Jan 14, 2010 at 11:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I think the dict configuration scheme will be a great addition to the
logging module. :)

Schiavo
Simon


From p.f.moore at gmail.com  Thu Jan 14 12:22:09 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 Jan 2010 11:22:09 +0000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>

2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I've no immediate need for the feature, but it would be good to have
something like this, so I'm +1.
Paul.


From ncoghlan at gmail.com  Thu Jan 14 12:46:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:46:19 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
Message-ID: <4B4F040B.8010607@gmail.com>

Paul Moore wrote:
> 2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> So, can you please indicate your vote for or against incorporating PEP 391 into Python?
> 
> I've no immediate need for the feature, but it would be good to have
> something like this, so I'm +1.

I'm in the same boat as Paul, but PEP 291 provides a nice forward
compatible configuration scheme that will work with any application
configuration approach that can produce an appropriate Python dictionary.

So +1 from me too - I think the PEP has now taken this as far as
theorising can go, and the only way to learn anything further is to put
it into practice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Jan 14 12:57:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:57:55 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4B4F06C3.7030806@gmail.com>

Vinay Sajip wrote:
> In October 2009 I created PEP 391 to propose a new method of
> configuring logging using dictionaries:
> 
> http://www.python.org/dev/peps/pep-0391/

Although one minor comment: you can probably remove the note about the
"ext://" and "cfg://" prefixes being provisional at this stage. Those
names look fine to me, so I don't see any point inviting a late-breaking
bikeshed discussion about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Thu Jan 14 14:53:29 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 08:53:29 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>

On Thu, Jan 14, 2010 at 4:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:
>
> ?http://www.python.org/dev/peps/pep-0391/
>
> In November 2009 I posted to this list that the PEP was ready for review.
>
> I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.
>
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.
>
> Thanks & regards,
>
> Vinay Sajip

I'm generally +1 - but given I know that Django 1.2 is slated to
implement something somewhat similar, I'm interested to hear how this
proposal meshes with their plan(s).

jesse


From vinay_sajip at yahoo.co.uk  Thu Jan 14 15:08:24 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 14:08:24 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
Message-ID: <92210.91656.qm@web25806.mail.ukl.yahoo.com>

> From: Jesse Noller <jnoller at gmail.com>

> I'm generally +1 - but given I know that Django 1.2 is slated to
> implement something somewhat similar, I'm interested to hear how this
> proposal meshes with their plan(s)..

Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:

http://groups.google.com/group/django-developers/msg/4ef81a2160257221

They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).

Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

Regards,

Vinay Sajip


      



From ssteinerx at gmail.com  Thu Jan 14 15:54:27 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 14 Jan 2010 09:54:27 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <62E362ED-5DD7-4D42-B5E0-B263A137FD5C@gmail.com>


On Jan 14, 2010, at 9:08 AM, Vinay Sajip wrote:

>> From: Jesse Noller <jnoller at gmail.com>
> 
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
> 
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
> 
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
> 
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
> 
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

That puts a huge +1 on there for me.  

If we can get this approved and have Django's logging improved *and* avoid a whole pile of duplicate effort in Django that's a huge win.

S



From jnoller at gmail.com  Thu Jan 14 15:56:18 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 09:56:18 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140656w68c7290agccfe7282cf96f2fb@mail.gmail.com>

On Thu, Jan 14, 2010 at 9:08 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> From: Jesse Noller <jnoller at gmail.com>
>
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
>
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
>
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
>
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
>
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.
>
> Regards,
>
> Vinay Sajip
>

Cool, +1 then :)


From mal at egenix.com  Thu Jan 14 20:19:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 20:19:12 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <4B4F6E30.50400@egenix.com>

Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users won't have PYTHONPATH or any other Python environment
variables setup.

Really, if you know that you are going to run Python 3 instead of
Python 2 or vice-versa it's easy enough to run

. py3env.sh
or
. py2env.sh

in order to setup your shell environment, much like you typically
do when using virtual Python installations or work on different
projects that require different setups.

If you just want to separate Python 2 and 3 files, use the
user site-packages dir which includes the Python version.

More experienced users could also write their own environment
switching sitecustomize.py or usercustomize.py.

And then there's the hackery option for experts that love
dirty tricks: add a .pth file which includes an "import mysetup"
line to fix-up you path and other settings.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Jan 14 22:02:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 15 Jan 2010 07:02:28 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <4B4F8664.4080107@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Guido van Rossum wrote:
>>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>>> just removing the antiquated use of environment variables altogether
>>> from Python 3 and avoid the issue completely.
>>>
>>> -1. They have their use, but more in controlled situations. If you
>>> have "global" env vars that you only want to use with Python 2.x,
>>> write a wrapper for Python 3 that invokes it with -E.
>> Perhaps a case can be made for Python 3 to assume -E by default, with a
>> -e option to enable reading of the environment variables?
>>
>> That way naive users could run Python3 without tripping over existing
>> Python2 environment variables, while other tools could readily set up a
>> different environment before launching Python3.
> 
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.

I was actually thinking of the situation where the OS had a preinstalled
Python 2 with a default PYTHONPATH setting and the user was playing with
Python 3.

However, I agree that that is a fairly unlikely scenario (since
preinstalled Pythons tend not to rely on the environment variables and a
user sophisticated enough to be playing with a new Python interpreter
should be able to cope with a few environment variable conflicts anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at voidspace.org.uk  Thu Jan 14 22:09:30 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 21:09:30 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F8664.4080107@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <4B4F8664.4080107@gmail.com>
Message-ID: <4B4F880A.5010809@voidspace.org.uk>

On 14/01/2010 21:02, Nick Coghlan wrote:
> However, I agree that that is a fairly unlikely scenario (since
> preinstalled Pythons tend not to rely on the e
Well, on the other hand I think that during the next few years it will 
be increasingly common for developers (and possibly users) to have 
Python 2 and Python 3 installed side-by-side.

Many libraries and applications may never make the jump to Python 3 and 
Python users may be using 'legacy' Python 2 code for many years to come. 
It will also become increasingly common for developers to be using 
Python 3 *primarily* and for Python 3 only libraries and applications to 
emerge.

Whilst there are workarounds we *are* in a situation that Python 2 and 
Python 3 share environment variables for the location of libraries and 
executing code on startup, whilst at the same time they are largely 
incompatible and need separate library paths and startup code.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ralf at brainbot.com  Thu Jan 14 22:25:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 14 Jan 2010 22:25:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com> (M.'s message of "Thu, 14 Jan 2010
	20:19:12 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <871vhswf90.fsf@brainbot.com>

"M.-A. Lemburg" <mal at egenix.com> writes:

>
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.
>
> Really, if you know that you are going to run Python 3 instead of
> Python 2 or vice-versa it's easy enough to run

You don't even know that you're going to run python. I have 40 python
scripts in my /usr/bin directory. 

>
> . py3env.sh
> or
> . py2env.sh
>
> in order to setup your shell environment, much like you typically
> do when using virtual Python installations or work on different
> projects that require different setups.

No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.

>
> If you just want to separate Python 2 and 3 files, use the
> user site-packages dir which includes the Python version.

Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
install those programs in a user's home directory. There are still
people running python <2.6.


From mal at egenix.com  Thu Jan 14 22:51:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 22:51:07 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <871vhswf90.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <871vhswf90.fsf@brainbot.com>
Message-ID: <4B4F91CB.2060106@egenix.com>

Ralf Schmitt wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>
>> Naive users won't have PYTHONPATH or any other Python environment
>> variables setup.
>>
>> Really, if you know that you are going to run Python 3 instead of
>> Python 2 or vice-versa it's easy enough to run
> 
> You don't even know that you're going to run python. I have 40 python
> scripts in my /usr/bin directory. 
> 
>>
>> . py3env.sh
>> or
>> . py2env.sh
>>
>> in order to setup your shell environment, much like you typically
>> do when using virtual Python installations or work on different
>> projects that require different setups.
> 
> No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.
> 
>>
>> If you just want to separate Python 2 and 3 files, use the
>> user site-packages dir which includes the Python version.
> 
> Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
> install those programs in a user's home directory. There are still
> people running python <2.6.

Note that you only need to have a different PYTHONPATH
setups for Python 2.x/3.x if you plan to run code that:

 * is not installed in site-packages (most OS shipped code
   will be found in the resp. system site-packages/ dir)

 * is not installed in a user site-packages directory
   (user installed code will typically go there (*))

 * uses modules/packages that come in two different versions
   (one for Python 2.x and one for 3.x) and use the same name
   for both versions

 * needs to work in both Python 2.x and 3.x


(*) The method for installing code in user site-packages dir is
running:

    python setup.py install --user

instead of the standard

    python setup.py install

which install to the system-wide site-packages.

BTW: I guess the bzr and mercurial wikis will need to be updated
accordingly - at least for users of Python >=2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From martin at v.loewis.de  Thu Jan 14 22:55:04 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Jan 2010 22:55:04 +0100
Subject: [Python-Dev] [ANN] Python 2.5.5 Release Candidate 1.
Message-ID: <4B4F92B8.30806@v.loewis.de>

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.5.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.6 (which is
2.6.4 at this point).

This releases fixes issues with the logging and tarfile modules, and
with thread-local variables. See the detailed release notes at the
website (also available as Misc/NEWS in the source distribution) for
details of bugs fixed.

For more information on Python 2.5.5, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.5.5

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

    http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
martin at v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)


From status at bugs.python.org  Fri Jan 15 18:07:26 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jan 2010 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100115170726.9963A7865F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/08/10 - 01/15/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2536 open (+32) / 16993 closed (+16) / 19529 total (+48)

Open issues with patches:  1024

Average duration of open issues: 710 days.
Median duration of open issues: 469 days.

Open Issues Breakdown
   open  2501 (+32)
pending    34 ( +0)

Issues Created Or Reopened (53)
_______________________________

PYTHON3PATH environment variable to supersede PYTHONPATH for mul 01/13/10
CLOSED http://bugs.python.org/issue2375    reopened r.david.murray                
                                                                               

Mark the compiler package as deprecated                          01/13/10
       http://bugs.python.org/issue6837    reopened ezio.melotti                  
                                                                               

test_distutils failure                                           01/15/10
CLOSED http://bugs.python.org/issue6961    reopened flox                          
       buildbot                                                                

test_urllib: unsetting missing 'env' variable                    01/08/10
CLOSED http://bugs.python.org/issue7026    reopened flox                          
                                                                               

Two float('nan') are not equal                                   01/08/10
CLOSED http://bugs.python.org/issue7660    created  jmfauth                       
                                                                               

compiling ctypes fails with non-ascii path                       01/08/10
       http://bugs.python.org/issue7661    created  pitrou                        
       patch                                                                   

time.utcoffset()                                                 01/09/10
       http://bugs.python.org/issue7662    created  techtonik                     
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points  01/10/10
CLOSED http://bugs.python.org/issue7663    created  exarkun                       
                                                                               

python logger does not handle IOError Exception                  01/10/10
CLOSED http://bugs.python.org/issue7664    created  yateenjoshi                   
                                                                               

test_urllib2 and test_ntpath fail if path contains "\"           01/10/10
       http://bugs.python.org/issue7665    created  flox                          
                                                                               

test_lib2to3 fails if path contains space                        01/10/10
CLOSED http://bugs.python.org/issue7666    created  flox                          
                                                                               

test_doctest fails with non-ascii path                           01/10/10
       http://bugs.python.org/issue7667    created  flox                          
       buildbot                                                                

test_httpservers fails with non-ascii path                       01/10/10
       http://bugs.python.org/issue7668    created  flox                          
       buildbot                                                                

test_unicode_file fails with non-ascii path                      01/10/10
CLOSED http://bugs.python.org/issue7669    created  flox                          
                                                                               

_sqlite3: Block *all* operations on a closed Connection object   01/10/10
       http://bugs.python.org/issue7670    created  haypo                         
       patch                                                                   

test_popen fails if path contains special char like ";"          01/11/10
       http://bugs.python.org/issue7671    reopened flox                          
       patch                                                                   

_ssl module causes segfault                                      01/10/10
       http://bugs.python.org/issue7672    created  ssoria                        
       patch                                                                   

audioop: check that length is a multiple of the size             01/11/10
       http://bugs.python.org/issue7673    created  haypo                         
       patch                                                                   

select.select() corner cases: duplicate fds, out-of-range fds    01/11/10
       http://bugs.python.org/issue7674    created  cdleary                       
                                                                               

help() doesn't accept unicode literals in built in docstrings    01/11/10
       http://bugs.python.org/issue7675    created  psd                           
       patch                                                                   

IDLE shell shouldn't use TABs                                    01/11/10
       http://bugs.python.org/issue7676    created  cben                          
                                                                               

distutils, better error message for setup.py upload -sign withou 01/11/10
       http://bugs.python.org/issue7677    created  illume                        
                                                                               

subprocess.Popen pipeline example code in the documentation is l 01/11/10
       http://bugs.python.org/issue7678    created  steven.k.wong                 
                                                                               

Warning building 2.7 on OS X 10.6 libintl.h "Present But Cannot  01/12/10
       http://bugs.python.org/issue7679    created  ssteinerX                     
                                                                               

pythonw crash while attempting to start() a thread object        01/12/10
       http://bugs.python.org/issue7680    created  dontbugme                     
                                                                               

Incorrect division in [wave.py]                                  01/12/10
CLOSED http://bugs.python.org/issue7681    created  alfps                         
       patch, needs review                                                     

Optimisation of if with constant expression                      01/12/10
       http://bugs.python.org/issue7682    created  sdefresne                     
       patch                                                                   

Wrong link in HTML documentation                                 01/12/10
CLOSED http://bugs.python.org/issue7683    created  francescor                    
                                                                               

decimal.py: infinity coefficients in tuples                      01/12/10
       http://bugs.python.org/issue7684    created  skrah                         
                                                                               

minor typo in re docs                                            01/12/10
CLOSED http://bugs.python.org/issue7685    created  mikejs                        
       patch                                                                   

redundant open modes 'rbb', 'wbb', 'abb' no longer work on Windo 01/13/10
       http://bugs.python.org/issue7686    created  ivank                         
                                                                               

Bluetooth support untested                                       01/13/10
       http://bugs.python.org/issue7687    created  pitrou                        
                                                                               

TypeError: __name__ must be set to a string object               01/13/10
       http://bugs.python.org/issue7688    created  frankmillman                  
                                                                               

Pickling of classes with a metaclass and copy_reg                01/13/10
       http://bugs.python.org/issue7689    created  craigcitro                    
       patch                                                                   

Wrong PEP number in importlib module manual page                 01/13/10
CLOSED http://bugs.python.org/issue7690    created  francescor                    
                                                                               

test_bsddb3 files are not always removed when test fails         01/13/10
CLOSED http://bugs.python.org/issue7691    created  flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero              01/13/10
CLOSED http://bugs.python.org/issue7692    created  Bugger                        
                                                                               

tarfile.extractall can't have unicode extraction path            01/13/10
       http://bugs.python.org/issue7693    created  pbienst                       
                                                                               

DeprecationWarning from distuils.commands.build_ext needs stackl 01/13/10
       http://bugs.python.org/issue7694    created  tseaver                       
       patch                                                                   

missing termios constants                                        01/13/10
       http://bugs.python.org/issue7695    created  conorh                        
                                                                               

Improve Memoryview/Buffer documentation                          01/13/10
       http://bugs.python.org/issue7696    created  flox                          
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes  01/13/10
CLOSED http://bugs.python.org/issue7697    created  flox                          
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame  01/14/10
CLOSED http://bugs.python.org/issue7698    created  exarkun                       
       patch                                                                   

strptime, strftime documentation                                 01/14/10
       http://bugs.python.org/issue7699    created  mikejs                        
       patch                                                                   

"-3" flag does not work anymore                                  01/14/10
CLOSED http://bugs.python.org/issue7700    created  flox                          
       patch                                                                   

fix output string length for binascii.b2a_uu()                   01/14/10
CLOSED http://bugs.python.org/issue7701    created  haypo                         
       patch                                                                   

Wrong order of parameters of _get_socket in SMTP class in smtpli 01/14/10
       http://bugs.python.org/issue7702    created  alito                         
                                                                               

Replace buffer()-->memoryview() in Lib/ctypes/test/              01/14/10
       http://bugs.python.org/issue7703    created  flox                          
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE         01/14/10
CLOSED http://bugs.python.org/issue7704    created  DhaReaL                       
                                                                               

libpython2.6.so is not linked correctly on FreeBSD when threads  01/15/10
       http://bugs.python.org/issue7705    created  aballier                      
       patch, needs review                                                     

Missing #include guards                                          01/15/10
       http://bugs.python.org/issue7706    created  akrpic77                      
       patch                                                                   

multiprocess.Queue operations during import can lead to deadlock 01/15/10
       http://bugs.python.org/issue7707    created  kripken                       
                                                                               

Conversion of longs to bytes and vice-versa.                     01/11/10
       http://bugs.python.org/issue1023290 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (67)
______________________

segfault in curses when calling redrawwin() before refresh()      825 days
       http://bugs.python.org/issue1266    mark.dickinson                
                                                                               

"RuntimeError: dictionary changed size during iteration" in Tkin  751 days
       http://bugs.python.org/issue1658    flox                          
       patch                                                                   

Exception exceptions.AttributeError '_shutdown' in <module 'thre  718 days
       http://bugs.python.org/issue1947    nadiasvertex                  
                                                                               

Backport dictviews to 2.7                                         713 days
       http://bugs.python.org/issue1967    alexandre.vassalotti          
       patch, needs review                                                     

Backport set and dict comprehensions                              665 days
       http://bugs.python.org/issue2333    alexandre.vassalotti          
       patch, 26backport                                                       

Backport set literals                                             663 days
       http://bugs.python.org/issue2335    alexandre.vassalotti          
       patch, 26backport                                                       

PYTHON3PATH environment variable to supersede PYTHONPATH for mul    1 days
       http://bugs.python.org/issue2375    lemburg                       
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     625 days
       http://bugs.python.org/issue2698    cmcqueen1975                  
                                                                               

arg 2 of PyErr_SetFromErrnoWithFilename should be const           619 days
       http://bugs.python.org/issue2758    brian.curtin                  
                                                                               

Trunk build issues in DragonFly BSD                               613 days
       http://bugs.python.org/issue2797    brian.curtin                  
       patch, needs review                                                     

Gzip cannot handle zero-padded output + patch                     610 days
       http://bugs.python.org/issue2846    pitrou                        
       patch, needs review                                                     

block operation on closed socket/pipe for multiprocessing         552 days
       http://bugs.python.org/issue3311    haypo                         
       patch, needs review                                                     

Add gamma function, error functions and other C99 math.h functio  543 days
       http://bugs.python.org/issue3366    mark.dickinson                
       patch, needs review                                                     

Macros for PyLong: sign, number of digits, fits in an int         425 days
       http://bugs.python.org/issue4294    mark.dickinson                
       patch                                                                   

reject unicode in zlib                                            379 days
       http://bugs.python.org/issue4757    haypo                         
       patch                                                                   

Distutils inappropriately reuses .o files between extension modu  320 days
       http://bugs.python.org/issue5372    tarek                         
       patch, patch, needs review                                              

Problem with email.MIME* library, using wrong new line            298 days
       http://bugs.python.org/issue5525    r.david.murray                
                                                                               

os.path.normpath doesn't preserve unicode                         263 days
       http://bugs.python.org/issue5827    ezio.melotti                  
       patch                                                                   

smtplib exception smtp.connect TypeError encode_plain             173 days
       http://bugs.python.org/issue6523    r.david.murray                
                                                                               

email.parser clips trailing \n of multipart/mixed part if part e  152 days
       http://bugs.python.org/issue6681    r.david.murray                
       patch                                                                   

Optimize PyBytes_FromObject.                                      151 days
       http://bugs.python.org/issue6688    alexandre.vassalotti          
       patch                                                                   

in xmlrpclib.py: NameError: global name 'HTTPSConnection' is not  143 days
       http://bugs.python.org/issue6769    krisvale                      
                                                                               

UnicodeDecodeError when retrieving binary data from cgi.FieldSto  126 days
       http://bugs.python.org/issue6854    r.david.murray                
                                                                               

test_distutils failure                                              0 days
       http://bugs.python.org/issue6961    flox                          
       buildbot                                                                

tmpnam should not be used if tempfile or mkstemp are available    110 days
       http://bugs.python.org/issue6965    flox                          
                                                                               

test_urllib: unsetting missing 'env' variable                       0 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

distutils and IronPython compatibility                             73 days
       http://bugs.python.org/issue7071    tarek                         
       26backport                                                              

weak dict iterators are fragile because of unpredictable GC runs   89 days
       http://bugs.python.org/issue7105    pitrou                        
       patch                                                                   

email:  msg.items() returns different values before and after ms   89 days
       http://bugs.python.org/issue7119    r.david.murray                
       patch                                                                   

get_payload(decode=True) eats last newline                         87 days
       http://bugs.python.org/issue7143    r.david.murray                
                                                                               

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo   49 days
       http://bugs.python.org/issue7382    alexandre.vassalotti          
       patch, easy                                                             

Document inspect.get(full)argspec limitation to Python function    39 days
       http://bugs.python.org/issue7422    georg.brandl                  
                                                                               

Py3.1: Fatal Python Error: Py_Initialize...unknown encoding: chc   35 days
       http://bugs.python.org/issue7441    georg.brandl                  
                                                                               

when piping output between subprocesses some fd is left open blo   36 days
       http://bugs.python.org/issue7448    flox                          
                                                                               

Solaris SPARC: _multiprocessing.so: symbol sem_timedwait: refere   35 days
       http://bugs.python.org/issue7454    srid                          
                                                                               

cPickle: stack underflow in load_pop()                             33 days
       http://bugs.python.org/issue7455    haypo                         
       patch                                                                   

crash in str.rfind() with an invalid start value                   33 days
       http://bugs.python.org/issue7458    haypo                         
       patch                                                                   

test_multiprocessing test_rapid_restart fails if port 9999 alrea   27 days
       http://bugs.python.org/issue7498    r.david.murray                
       patch, buildbot                                                         

Extended slicing with classic class behaves strangely               3 days
       http://bugs.python.org/issue7532    mark.dickinson                
       patch                                                                   

stringlib fastsearch could be improved on 64-bit builds            13 days
       http://bugs.python.org/issue7607    pitrou                        
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option    7 days
       http://bugs.python.org/issue7617    tarek                         
       patch                                                                   

[patch] improve unicode methods: split() rsplit() and replace()    10 days
       http://bugs.python.org/issue7622    pitrou                        
       patch                                                                   

bytearray needs more tests for "b.some_method()[0] is not b"       10 days
       http://bugs.python.org/issue7625    pitrou                        
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                    4 days
       http://bugs.python.org/issue7648    orsenthil                     
       easy                                                                    

[patch] Enable additional bytes and memoryview tests.               5 days
       http://bugs.python.org/issue7654    pitrou                        
       patch                                                                   

test_ctypes failure on AIX 5.3                                      4 days
       http://bugs.python.org/issue7657    mallayya                      
       patch                                                                   

Two float('nan') are not equal                                      0 days
       http://bugs.python.org/issue7660    mark.dickinson                
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points     1 days
       http://bugs.python.org/issue7663    lemburg                       
                                                                               

python logger does not handle IOError Exception                     1 days
       http://bugs.python.org/issue7664    vinay.sajip                   
                                                                               

test_lib2to3 fails if path contains space                           0 days
       http://bugs.python.org/issue7666    benjamin.peterson             
                                                                               

test_unicode_file fails with non-ascii path                         2 days
       http://bugs.python.org/issue7669    ezio.melotti                  
                                                                               

Incorrect division in [wave.py]                                     1 days
       http://bugs.python.org/issue7681    benjamin.peterson             
       patch, needs review                                                     

Wrong link in HTML documentation                                    0 days
       http://bugs.python.org/issue7683    ezio.melotti                  
                                                                               

minor typo in re docs                                               0 days
       http://bugs.python.org/issue7685    ezio.melotti                  
       patch                                                                   

Wrong PEP number in importlib module manual page                    0 days
       http://bugs.python.org/issue7690    brett.cannon                  
                                                                               

test_bsddb3 files are not always removed when test fails            2 days
       http://bugs.python.org/issue7691    flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero                 0 days
       http://bugs.python.org/issue7692    mark.dickinson                
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes     0 days
       http://bugs.python.org/issue7697    pjenvey                       
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame     0 days
       http://bugs.python.org/issue7698    skip.montanaro                
       patch                                                                   

"-3" flag does not work anymore                                     1 days
       http://bugs.python.org/issue7700    brett.cannon                  
       patch                                                                   

fix output string length for binascii.b2a_uu()                      1 days
       http://bugs.python.org/issue7701    pitrou                        
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE            0 days
       http://bugs.python.org/issue7704    tim_one                       
                                                                               

Support for sending multipart form data                          2452 days
       http://bugs.python.org/issue727898  r.david.murray                
                                                                               

email.MIME*.as_string removes duplicate spaces                   1440 days
       http://bugs.python.org/issue1422094 r.david.murray                
       easy                                                                    

test_parsedate_acceptable_to_time_functions+DST == :-(           1394 days
       http://bugs.python.org/issue1454285 r.david.murray                
                                                                               

email module does not complay with RFC 2046: CRLF issue          1196 days
       http://bugs.python.org/issue1571841 r.david.murray                
                                                                               

email.FeedParser.BufferedSubFile improperly handles "\r\n"        969 days
       http://bugs.python.org/issue1721862 r.david.murray                
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 20 dtoa.c: oversize b in quorem                                      11 days
open    http://bugs.python.org/issue7632   

 18 _ssl module causes segfault                                        5 days
open    http://bugs.python.org/issue7672   

 12 Speed up cPickle's pickling generally                            287 days
open    http://bugs.python.org/issue5683   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    7 days
open    http://bugs.python.org/issue7658   

  8 Fatal error on thread creation in low memory condition            27 days
open    http://bugs.python.org/issue7544   

  8 Direct calls to PyObject_Del/PyObject_DEL are broken for --with  558 days
open    http://bugs.python.org/issue3299   

  7 "-3" flag does not work anymore                                    1 days
closed  http://bugs.python.org/issue7700   

  7 tarfile.extractall can't have unicode extraction path              2 days
open    http://bugs.python.org/issue7693   

  7 test_urllib: unsetting missing 'env' variable                      0 days
closed  http://bugs.python.org/issue7026   

  7 os.path.abspath with unicode argument should call os.getcwdu     542 days
open    http://bugs.python.org/issue3426   




From g.brandl at gmx.net  Sat Jan 16 21:05:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 16 Jan 2010 21:05:46 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<20100113200840.GC14858@phd.pp.ru>
	<319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
Message-ID: <hit67o$7v4$1@ger.gmane.org>

Am 13.01.2010 21:27, schrieb Lennart Regebro:
> On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
>> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>>> What do you need to do in the PYTHONSTARTUP file?
>>> Ten years of Python programming, and I didn't even know it existed. :-)
>>
>>   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.
> 
> Cheese and fries! :-)
> 
> Anyway, I don't see how this could pose any problems, because any user
> advanced enough to do these things will be advanced enough to
> understand what the problem is and fix it so it works under Python 3
> too.

I'd propose adding a bit of text to the environment variables documentation,
and especially about the needs of PYTHONSTARTUP files.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From asmodai at in-nomine.org  Sat Jan 16 21:50:56 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 16 Jan 2010 21:50:56 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <874ompg225.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
Message-ID: <20100116205056.GL99670@nexus.in-nomine.org>

-On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>hehe. tab completion:

With bpython and ipython available, why would you even want to stick to the
'plain old' interactive interpreter?

(Sorry to derail the discussion, but maybe there's more people that have not
heard of either or both.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For ever, brother, hail and farewell...


From solipsis at pitrou.net  Sat Jan 16 21:57:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jan 2010 20:57:13 +0000 (UTC)
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <loom.20100116T215639-769@post.gmane.org>

Jeroen Ruigrok van der Werven <asmodai <at> in-nomine.org> writes:
> 
> -On [20100113 22:13], Ralf Schmitt (ralf <at> brainbot.com) wrote:
> >hehe. tab completion:
> 
> With bpython and ipython available, why would you even want to stick to the
> 'plain old' interactive interpreter?

Why wouldn't we?
There are probably many more people using the standard Python prompt than
ipython.





From ben+python at benfinney.id.au  Sat Jan 16 23:41:03 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2010 09:41:03 +1100
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <87y6jxk70g.fsf@benfinney.id.au>

Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:

> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
> >hehe. tab completion:
>
> With bpython and ipython available, why would you even want to stick
> to the 'plain old' interactive interpreter?

Because those optional extras are not always available, whereas the
standard interactive interpreter is always available with CPython
distributions.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Sun Jan 17 01:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:22:10 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87y6jxk70g.fsf@benfinney.id.au>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<874ompg225.fsf@brainbot.com>	<20100116205056.GL99670@nexus.in-nomine.org>
	<87y6jxk70g.fsf@benfinney.id.au>
Message-ID: <4B525832.8090904@gmail.com>

Ben Finney wrote:
> Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:
> 
>> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>>> hehe. tab completion:
>> With bpython and ipython available, why would you even want to stick
>> to the 'plain old' interactive interpreter?
> 
> Because those optional extras are not always available, whereas the
> standard interactive interpreter is always available with CPython
> distributions.

I've never even contemplated the idea of installing 3rd party apps for
the 4 current development builds (2.x trunk, 2.x maintenance, 3.x trunk,
3.x maintenance) on my home development machine.

And of course work suffers from the problem of not being allowed to
install arbitrary apps I downloaded from the internet without getting
the licensing vetted for commercial acceptability.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 01:34:08 2010
From: nad at acm.org (Ned Deily)
Date: Sat, 16 Jan 2010 16:34:08 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
Message-ID: <nad-79FC16.16340816012010@news.gmane.org>

I've recently seen a couple of references to 3.1.2 go by in checkins 
which made me wonder whether dates have been proposed yet for updates to 
either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
references in the PEPs.  Some advance warning would be nice.  I assume 
that some critical problem could trigger planning for an update on short 
notice; is there a time-limit trigger as well?

-- 
 Ned Deily,
 nad at acm.org



From ncoghlan at gmail.com  Sun Jan 17 01:55:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:55:38 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52600A.7060201@gmail.com>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Usually there is one more regular maintenance release of the existing
maintenance branches within a few months of the creation of the next
version (releases before then are at the discretion of the release
manager, and security releases continue for much longer).

So take the planned 2.7 and 3.2 release dates and add a couple of months
to each one to get the likely release dates for the 2.6.x and 3.1.x
maintenance releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Sun Jan 17 10:21:49 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Jan 2010 10:21:49 +0100
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52D6AD.6090302@v.loewis.de>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Barry was once proposing time-based releases; this idea didn't really
catch on.

It's the release manager who decides when the next bug fix release will
be made, often in response to developers asking for one.

Regards,
Martin



From solipsis at pitrou.net  Sun Jan 17 14:00:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 13:00:04 +0000 (UTC)
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <loom.20100117T135910-313@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.

There are a couple of release blockers right now. Once they are fixed or
deferred, I think it would be nice to have a 3.1.2.
Why do you need "some advance warning" though?




From ncoghlan at gmail.com  Sun Jan 17 14:40:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 23:40:42 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <loom.20100117T135910-313@post.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
Message-ID: <4B53135A.7060104@gmail.com>

Antoine Pitrou wrote:
> Ned Deily <nad <at> acm.org> writes:
>> I've recently seen a couple of references to 3.1.2 go by in
>> checkins which made me wonder whether dates have been proposed yet
>> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
>> didn't see any references in the PEPs.  Some advance warning would
>> be nice.
> 
> There are a couple of release blockers right now. Once they are fixed
> or deferred, I think it would be nice to have a 3.1.2. Why do you
> need "some advance warning" though?

Advance warning does allow interested users that would consider
upgrading to schedule time for testing before the maintenance release
comes out. This is particularly useful in helping to make a 1-week RC
period effective in picking up issues that might otherwise lead to a
brown paper bag release to fix major issues that slipped through our own
automated test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 19:01:37 2010
From: nad at acm.org (Ned Deily)
Date: Sun, 17 Jan 2010 10:01:37 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
	<4B53135A.7060104@gmail.com>
Message-ID: <nad-25165C.10013717012010@ger.gmane.org>

In article <4B53135A.7060104 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> I've recently seen a couple of references to 3.1.2 go by in
> >> checkins which made me wonder whether dates have been proposed yet
> >> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
> >> didn't see any references in the PEPs.  Some advance warning would
> >> be nice.
> > There are a couple of release blockers right now. Once they are fixed
> > or deferred, I think it would be nice to have a 3.1.2. Why do you
> > need "some advance warning" though?
> 
> Advance warning does allow interested users that would consider
> upgrading to schedule time for testing before the maintenance release
> comes out. This is particularly useful in helping to make a 1-week RC
> period effective in picking up issues that might otherwise lead to a
> brown paper bag release to fix major issues that slipped through our own
> automated test coverage.

That. and resource contention: there are always potential fixes in the 
pipeline that could or should be bumped in priority if one knows there 
is a code cutoff approaching.

-- 
 Ned Deily,
 nad at acm.org



From ziade.tarek at gmail.com  Sun Jan 17 20:51:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 20:51:58 +0100
Subject: [Python-Dev] Enhancing the shutil module
Message-ID: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>

Hello,

For 2.7/3.2, I am in the process of removing modules in Distutils that
can be replaced by calls to existing functions in stdlib. For
instance, "dir_util" and "file_util" (old modules from the Python 1.x
era) are going away in favor of calls to shutil (and os), so the
Distutils package gets lighter.

Another module I would like to move away from Distutils is
"archive_util". It contains helpers to build archives, whether they
are zip or tar files. I propose to move those useful functions into
shutil, as this seems the most logical place.

I also propose to maintain this "shutil" module for now on (no one is
declared as a maintainer in maintainers.rst) since Distutils will
become a heavy user of its functions.

Any objections/opinions ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 20:53:03 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 19:53:03 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <4B536A9F.5050203@voidspace.org.uk>

On 17/01/2010 19:51, Tarek Ziad? wrote:
> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>
> Regards,
> Tarek
>
>    
I think it's a great idea. :-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From brett at python.org  Sun Jan 17 20:55:47 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jan 2010 11:55:47 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>

On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>

If it's archive-agnostic then shutil is probably the best place.


>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
>
Great!


> Any objections/opinions ?
>

No objections from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100117/f97ac6eb/attachment-0004.htm>

From solipsis at pitrou.net  Sun Jan 17 21:04:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 20:04:41 +0000 (UTC)
Subject: [Python-Dev] Enhancing the shutil module
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <loom.20100117T210307-719@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.

Are these functions portable? Do they rely on external programs?

> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.

It's ok for me.

Regards

Antoine.




From ziade.tarek at gmail.com  Sun Jan 17 21:09:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:09:18 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
Message-ID: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>

On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Hello,
>>
>> For 2.7/3.2, I am in the process of removing modules in Distutils that
>> can be replaced by calls to existing functions in stdlib. For
>> instance, "dir_util" and "file_util" (old modules from the Python 1.x
>> era) are going away in favor of calls to shutil (and os), so the
>> Distutils package gets lighter.
>>
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>
> If it's archive-agnostic then shutil is probably the best place.

In more details:
It allows the creation of gzip, bzip2, tar and zip files through a single API.
There's a registry of supported formats and the API is driven by a
format identifier.

To do the work it uses stdlib's compression modules. Although it tries
the "zip" system command as a fallback if the "zipfile" module is not
present.

(notice that I've removed the support of "compress" (.Z) some time ago)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 21:15:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 20:15:00 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <loom.20100117T210307-719@post.gmane.org>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
Message-ID: <4B536FC4.9090304@voidspace.org.uk>

On 17/01/2010 20:04, Antoine Pitrou wrote:
> Tarek Ziad?<ziade.tarek<at>  gmail.com>  writes:
>    
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>>      
> Are these functions portable? Do they rely on external programs?
>
>    

I believe that part of the work that Tarek has been doing has been to 
make these distutils commands use the Python standard library and not 
depend on external programs. In which case they seem like *excellent* 
additions to the shutil module.

Of course Tarek can speak for himself...

Michael

>> I also propose to maintain this "shutil" module for now on (no one is
>> declared as a maintainer in maintainers.rst) since Distutils will
>> become a heavy user of its functions.
>>      
> It's ok for me.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Sun Jan 17 21:43:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:43:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B536FC4.9090304@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
Message-ID: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:15 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>> Are these functions portable? Do they rely on external programs?
>>
>>
>
> I believe that part of the work that Tarek has been doing has been to make
> these distutils commands use the Python standard library and not depend on
> external programs. In which case they seem like *excellent* additions to the
> shutil module.

yes, in the past the "tar" files where created using the "tar" command
but this has been changed. For a while now, they are portable and they
use stdlib code only. A recent addition is to be able to define
user/group permissions in the tar files, thanks to Lars' work in the
tarfile module.

There's one remaining external call for "zip" done if the zip module
is not found, but I am happy to remove it and throw an exception if
it's not found, and keep the external "zip" call on Distutils side, so
shutil stays 100% stdlib-powered.

> Of course Tarek can speak for himself...

Thanks for explaining it ! :)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From sridharr at activestate.com  Sun Jan 17 22:50:52 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 17 Jan 2010 13:50:52 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
Message-ID: <4B53863C.5060304@activestate.com>

On 1/17/2010 12:09 PM, Tarek Ziad? wrote:
> On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon<brett at python.org>  wrote:
>> >  On Sun, Jan 17, 2010 at 11:51, Tarek Ziad?<ziade.tarek at gmail.com>  wrote:
>>> >>  Another module I would like to move away from Distutils is
>>> >>  "archive_util". It contains helpers to build archives, whether they
>>> >>  are zip or tar files. I propose to move those useful functions into
>>> >>  shutil, as this seems the most logical place.
>> >  If it's archive-agnostic then shutil is probably the best place.
> In more details:
> It allows the creation of gzip, bzip2, tar and zip files through a single API.
> There's a registry of supported formats and the API is driven by a
> format identifier.

Will it also allow decompression of the said archive types? Distribute 
has some utility code to handle zip/tar archives. So does PyPM. This is 
because the `tarfile` and `zipfile` modules do not "just work" due to 
several issues.

See http://gist.github.com/279606

Take note of the following in the above code:

  1) _ensure_read_write_access
  2) *File.is_valid
  3) ZippedFile.extract ... issue 6510
  4) ZippedFile.extract ... issue 6609
  5) TarredFile.extract ... issue 6584
  6) The way unpack() detects the unpacked directory.

-srid


From ziade.tarek at gmail.com  Sun Jan 17 23:09:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 23:09:29 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B53863C.5060304@activestate.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
	<4B53863C.5060304@activestate.com>
Message-ID: <94bdd2611001171409j4ec1e0bfj47e59894fdec0d8a@mail.gmail.com>

On Sun, Jan 17, 2010 at 10:50 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
[..]
> Will it also allow decompression of the said archive types?

No but it would make sense having this one as well.
Distribute/Setuptools' "unpack_archive" (used by easy_install) was
implemented using the same principle as Distutils' "make_archive".

I can add it as well while I am at it : it has been successfully used
for years by easy_install.

> Distribute has
> some utility code to handle zip/tar archives. So does PyPM. This is because
> the `tarfile` and `zipfile` modules do not "just work" due to several
> issues.
>
> See http://gist.github.com/279606
>
> Take note of the following in the above code:
>
> ?1) _ensure_read_write_access
> ?2) *File.is_valid
> ?3) ZippedFile.extract ... issue 6510
> ?4) ZippedFile.extract ... issue 6609
> ?5) TarredFile.extract ... issue 6584

Looks like some of these are already fixed (I just looked quickly in
the bug tracker though)
If its not already done and pending, it would be great if you could
refactor your fixes into patches for the remaining issues for each one
of those modules

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From barry at python.org  Sun Jan 17 23:56:37 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Jan 2010 17:56:37 -0500
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <4B52D6AD.6090302@v.loewis.de>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<4B52D6AD.6090302@v.loewis.de>
Message-ID: <BEC881FD-2BDE-4B5C-A570-B8C1E23D6DE1@python.org>

On Jan 17, 2010, at 4:21 AM, Martin v. L?wis wrote:

> Barry was once proposing time-based releases; this idea didn't really
> catch on.

I'm still a proponent of this, but it doesn't seem like there's enough support for it.

> It's the release manager who decides when the next bug fix release will
> be made, often in response to developers asking for one.

I'm happy to make a 2.6.5 release whenever it makes sense.
-Barry



From ncoghlan at gmail.com  Mon Jan 18 13:40:01 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Jan 2010 22:40:01 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
Message-ID: <4B5456A1.2080109@gmail.com>

Tarek Ziad? wrote:
> There's one remaining external call for "zip" done if the zip module
> is not found, but I am happy to remove it and throw an exception if
> it's not found, and keep the external "zip" call on Distutils side, so
> shutil stays 100% stdlib-powered.

+1 for that approach. These changes all sound like nice additions to
shutil, and hooray for every module that gets adopted by an active
maintainer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Mon Jan 18 14:34:14 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 14:34:14 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5456A1.2080109@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
Message-ID: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>

On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
> 
> Tarek Ziad? wrote:
>> There's one remaining external call for "zip" done if the zip module
>> is not found, but I am happy to remove it and throw an exception if
>> it's not found, and keep the external "zip" call on Distutils side, so
>> shutil stays 100% stdlib-powered.
> 
> +1 for that approach. These changes all sound like nice additions to
> shutil, and hooray for every module that gets adopted by an active
> maintainer :)

Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).

Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

From doug.hellmann at gmail.com  Mon Jan 18 14:46:05 2010
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 18 Jan 2010 08:46:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils  
>>> side, so
>>> shutil stays 100% stdlib-powered.
>>
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
>
> Isn't it a bit weird to include that to shutil though? shutil  
> advertises itself as "a number of high-level operations on files and  
> collections of files." and from what I understood it was a bunch of  
> shell-type utility functions to easily copy, move or remove files  
> and directories (that's pretty much all there is in it at this time).
>
> Wouldn't it make more sense to put those "archive utils" functions/ 
> objects in a new module separate from shutil, dealing specifically  
> with cross-archive APIs and linked from the current archive-specific  
> modules (essentially, just take the current archive_util, move it to  
> the toplevel of the stdlib and maybe rename it)? It would also make  
> the module much easier to find when searching through the module  
> listing, I think.

+1



From fuzzyman at voidspace.org.uk  Mon Jan 18 14:57:49 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 18 Jan 2010 13:57:49 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
Message-ID: <4B5468DD.5040200@voidspace.org.uk>

On 18/01/2010 13:46, Doug Hellmann wrote:
>
> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>
>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>
>>> Tarek Ziad? wrote:
>>>> There's one remaining external call for "zip" done if the zip module
>>>> is not found, but I am happy to remove it and throw an exception if
>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>> shutil stays 100% stdlib-powered.
>>>
>>> +1 for that approach. These changes all sound like nice additions to
>>> shutil, and hooray for every module that gets adopted by an active
>>> maintainer :)
>>
>> Isn't it a bit weird to include that to shutil though? shutil 
>> advertises itself as "a number of high-level operations on files and 
>> collections of files." 

Well - isn't what's being proposed "a number of high-level operations on 
files and collections of files." ?

>> and from what I understood it was a bunch of shell-type utility functions

like tar and zip? :-)

>> to easily copy, move or remove files and directories (that's pretty 
>> much all there is in it at this time).
>>

I don't think the additions are out of place prima-facie.

>> Wouldn't it make more sense to put those "archive utils" 
>> functions/objects in a new module separate from shutil, dealing 
>> specifically with cross-archive APIs and linked from the current 
>> archive-specific modules (essentially, just take the current 
>> archive_util, move it to the toplevel of the stdlib and maybe rename 
>> it)? It would also make the module much easier to find when searching 
>> through the module listing, I think.
>
> +1
>

Proliferation of modules is itself a bad thing though.

Michael


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Mon Jan 18 15:34:12 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 15:34:12 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <94bdd2611001180634sacd998fm6f67dc680e2e61c3@mail.gmail.com>

On Mon, Jan 18, 2010 at 2:57 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>>> Wouldn't it make more sense to put those "archive utils"
>>> functions/objects in a new module separate from shutil, dealing specifically
>>> with cross-archive APIs and linked from the current archive-specific modules
>>> (essentially, just take the current archive_util, move it to the toplevel of
>>> the stdlib and maybe rename it)? It would also make the module much easier
>>> to find when searching through the module listing, I think.
>>
>> +1
>>
>
> Proliferation of modules is itself a bad thing though.

I am with Michael here. I think having this function in shutil is the
right place.

For the find problem, I think docs.python.org is easy to search now,
as long as the shutil module documentation has an good set of examples
for this new API.

We could even add references in the tarfile, zipfile modules
documentation pointing to these examples.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org


From masklinn at masklinn.net  Mon Jan 18 15:56:05 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 15:56:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>

On 18 Jan 2010, at 14:57 , Michael Foord wrote:
> 
> On 18/01/2010 13:46, Doug Hellmann wrote:
>> 
>> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>> 
>>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>> 
>>>> Tarek Ziad? wrote:
>>>>> There's one remaining external call for "zip" done if the zip module
>>>>> is not found, but I am happy to remove it and throw an exception if
>>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>>> shutil stays 100% stdlib-powered.
>>>> 
>>>> +1 for that approach. These changes all sound like nice additions to
>>>> shutil, and hooray for every module that gets adopted by an active
>>>> maintainer :)
>>> 
>>> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." 
> 
> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
> 
Well no those are high-level operations on a very restricted set of file types (archives).

>>> and from what I understood it was a bunch of shell-type utility functions
> 
> like tar and zip? :-)
> 
Tar and zip have a module each at this time, so they're considered pretty big. I don't think anybody would consider "mv" big enough to give it a module on its own.

>>> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.
>> 
>> +1
>> 
> 
> Proliferation of modules is itself a bad thing though.
Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.

Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

From phd at phd.pp.ru  Mon Jan 18 16:03:38 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 18 Jan 2010 18:03:38 +0300
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <20100118150337.GA19391@phd.pp.ru>

On Mon, Jan 18, 2010 at 02:34:14PM +0100, Masklinn wrote:
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

   +1

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From ziade.tarek at gmail.com  Mon Jan 18 16:24:37 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 16:24:37 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
	<6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
Message-ID: <94bdd2611001180724u7f48e620ub32e13ef96c873b9@mail.gmail.com>

On Mon, Jan 18, 2010 at 3:56 PM, Masklinn <masklinn at masklinn.net> wrote:
[..]
>> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
>>
> Well no those are high-level operations on a very restricted set of file types (archives)

not really: make_archive/unpack_archive, are also dealing with files
and directories.

[..]
>> Proliferation of modules is itself a bad thing though.
> Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.
>
> Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

I am not sure why this would happen. For instance, shutil is already
on the top of the os module since a few major versions IIRC, because
it reads and writes files and directories. But it was not moved into
the os package (or vice-versa)

The shutil module uses APIs to read and write files. So if it works
with archives, it's just a specific read/write API that is used, but
that doesn't mean tarfile and zipfile might be reunited with shutil
imho.

If the shutil module is restricted to high-level files and directories
manipulation, working with archives is just a target like another I
think.

But at the end I am 0- to create a new module, because what really
matters to me is to take it out of Distutils :)

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From listsin at integrateddevcorp.com  Mon Jan 18 16:56:05 2010
From: listsin at integrateddevcorp.com (Steve Steiner (listsin))
Date: Mon, 18 Jan 2010 10:56:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>> 
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils side, so
>>> shutil stays 100% stdlib-powered.
>> 
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
> 
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

As much of a pain as it is to get new modules accepted, I agree that mixing archiving functions into shutil is not the right way to do it and that a separate archive_util module would make much more sense and would give a logical place to put any extensions to archive handling.

S





From rdmurray at bitdance.com  Mon Jan 18 20:28:47 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 18 Jan 2010 14:28:47 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
Message-ID: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>

On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
> As much of a pain as it is to get new modules accepted, I agree that
> mixing archiving functions into shutil is not the right way to do it
> and that a separate archive_util module would make much more sense and
> would give a logical place to put any extensions to archive handling.

Looking at the source code and API for both shutil and archive_util, I
think that the archive_util methods fit into shutil.  shutil currently
wraps some standard library facilities with convenience functions
for operations you might otherwise perform at the shell command line using
OS facilities.  As far as I can tell, archive_util does the same, and
seems quite within the shutil mission of "high level file operations".

So +1 from me for putting these in shutil.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From p.f.moore at gmail.com  Mon Jan 18 20:48:43 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 18 Jan 2010 19:48:43 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
Message-ID: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>

2010/1/18 R. David Murray <rdmurray at bitdance.com>:
> On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
>> As much of a pain as it is to get new modules accepted, I agree that
>> mixing archiving functions into shutil is not the right way to do it
>> and that a separate archive_util module would make much more sense and
>> would give a logical place to put any extensions to archive handling.
>
> Looking at the source code and API for both shutil and archive_util, I
> think that the archive_util methods fit into shutil. ?shutil currently
> wraps some standard library facilities with convenience functions
> for operations you might otherwise perform at the shell command line using
> OS facilities. ?As far as I can tell, archive_util does the same, and
> seems quite within the shutil mission of "high level file operations".
>
> So +1 from me for putting these in shutil.

Conceptually, I'm happy with these going into shutil (and +1 on the
rest of Tarek's proposal, too!)

To my mind, shutil is a module for higher-level operations on files -
the sort of things you'd do in shell commands, like move a batch of
files around (mv), create a directory tree (mkdir -p). Tarring or
zipping up a batch of files fits nicely into that space.

Paul.


From david.lyon at preisshare.net  Tue Jan 19 02:53:43 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 Jan 2010 20:53:43 -0500
Subject: [Python-Dev] PyCon Keynote
In-Reply-To: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
References: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
Message-ID: <0dfb370163cf3a284ca256f49662db74@preisshare.net>

On Wed, 13 Jan 2010 10:51:14 -0800, Guido van Rossum <guido at python.org>
wrote:
> Please mail me topics you'd like to hear me talk about in my keynote
> at PyCon this year.

As a typical (but perphaps more vocal) python developer I would
like to hear things that might aspire me and others to move to 
python 3.

The way I see it is that python 2.x represents everyones 
comfort zone. It's safe and nobody got fired at work for 
using python 2.x

I would like to hear how python 3 could be 'shaken' up
with a slightly less conservative policy that has gone
with the python 2.x series.

The logic perhaps being that since only a minority
use the 3.x series, it's functionality set is still
more up in the air. imo it needs bigger batteries
to give it more power than the 2.x series.

This meaning that the stdlib could take an extra
5-10 packages not in the 2.x series. Just as
one example, sphinx - a great documentation
tool. I can easily name five or six others.

I'd also like to hear more of your ideas on pypi
and getting it to have as much who-ha as CPAN.

You could do a lot to enlarge the developer
group. Help us all get our priorities to be
inline with your own wishes and expectations.

If you ask us all to put in a big year and
buy you a beer at the end.. I'm certain we
all will.

Every little bit of inspiration and direction
counts for a lot...

Best Regards

David











From ncoghlan at gmail.com  Tue Jan 19 12:20:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Jan 2010 21:20:05 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
	<79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
Message-ID: <4B559565.8090602@gmail.com>

Paul Moore wrote:
> 2010/1/18 R. David Murray <rdmurray at bitdance.com>:
>> So +1 from me for putting these in shutil.
> 
> Conceptually, I'm happy with these going into shutil (and +1 on the
> rest of Tarek's proposal, too!)
> 
> To my mind, shutil is a module for higher-level operations on files -
> the sort of things you'd do in shell commands, like move a batch of
> files around (mv), create a directory tree (mkdir -p). Tarring or
> zipping up a batch of files fits nicely into that space.

This is also reflected in the way at least Windows handles archives
these days - it took them a couple of iterations to get it right (and
resolve some of the performance impacts), but Explorer now does a decent
job of integrating archives into the directory tree as "folders that
happen to be compressed".

Are archives as fundamental as directories and files? No. But in the
context of shutil, the fact that their internal structure is largely
about directories and files makes them more than just another arbitrary
file type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 19 14:16:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 08:16:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
Message-ID: <20100119081639.5d431ed9@freewill>

I've just updated the Launchpad mirrors for the 4 active Python branches,
trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar branches
on code.python.org but it's probably been 7 months or so since those were
regularly updated.  Now the Launchpad branches sync against the read-only
Subversion branches at http://svn.python.org, so they should remain up-to-date
(within the re-sync timeframe of about 4 hours).

This means you can once again use Bazaar to get local branches of Python, and
you can of course push your own branches to Launchpad.  I believe you can even
use the bzr-svn plugin to commit changes back to the Subversion master, though
I have not yet tried this.

To get a local branch, just do any of the following:

    % bzr branch lp:python (for trunk)
    % bzr branch lp:python/2.6
    % bzr branch lp:python/py3k
    % bzr branch lp:python/3.1

(It's fairly easy to create new mirrors for other Subversion branches,
e.g. Python 2.5; just drop me an email if you want them.)

If you're going to create a lot of branches you probably want to put them in a
shared repository.  E.g.

    % bzr init-repo pythonbzr
    % cd pythonbzr
    % bzr branch lp:python/py3k

Bazaar 2.0 or better is recommended.  For me, it took about 5m to check the
first branch out from Launchpad, and then about 30s or so for each subsequent
branch.

Enjoy,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/89a42d75/attachment-0002.pgp>

From abpillai at gmail.com  Tue Jan 19 14:37:18 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Tue, 19 Jan 2010 19:07:18 +0530
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <8548c5f31001190537l2bcab5cfkb6f461910e4abd8b@mail.gmail.com>

On Mon, Jan 18, 2010 at 1:21 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>

+1 for this. Just make sure that you change the docstring of shutil
 which now reads as,

" shutil - Utility functions for copying files and directory trees."

 According to this "definition", archives don't fit in there. But the
 functionality does fit right in, so just need to make sure that it
 is reflected in the __doc__ .


> Regards,
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/55892759/attachment-0002.htm>

From vinay_sajip at yahoo.co.uk  Tue Jan 19 16:50:42 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 19 Jan 2010 15:50:42 +0000 (UTC)
Subject: [Python-Dev] Mailing List archive corruption?
Message-ID: <loom.20100119T164757-651@post.gmane.org>

Hi,

When I look at the mailing list archive for python-dev, I see some odd stuff at
the bottom of the page:

http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232

Anyone know what's happened?

Regards,

Vinay Sajip



From barry at python.org  Tue Jan 19 17:07:48 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:07:48 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <loom.20100119T164757-651@post.gmane.org>
References: <loom.20100119T164757-651@post.gmane.org>
Message-ID: <20100119110748.69bc564a@freewill>

On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:

>When I look at the mailing list archive for python-dev, I see some odd stuff at
>the bottom of the page:
>
>http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>
>Anyone know what's happened?

WTF?  I think the archives were recently regenerated, so there's probably a
fubar there.  CC'ing the postmasters.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/8947a2e9/attachment-0002.pgp>

From foom at fuhm.net  Tue Jan 19 17:24:57 2010
From: foom at fuhm.net (James Y Knight)
Date: Tue, 19 Jan 2010 11:24:57 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119110748.69bc564a@freewill>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
Message-ID: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>


On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:

> On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
>
>> When I look at the mailing list archive for python-dev, I see some  
>> odd stuff at
>> the bottom of the page:
>>
>> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>>
>> Anyone know what's happened?
>
> WTF?  I think the archives were recently regenerated, so there's  
> probably a
> fubar there.  CC'ing the postmasters.

That happens if messages had unescaped "From" lines in the middle of  
them.

No doubt, you've now broken every link anyone had ever made into the  
python-dev archives, because now all the article numbers are  
different. BTDT...unfortunately... Pipermail really is quite crappy,  
sigh.

Anyhow, when I did that, I went back to a backup to get the original  
article numbers, and edited the mbox file escaping From lines or  
adding additional empty messages until the newly regenerated article  
numbers matched the originals. I'd highly recommend going through that  
painful process, since I suspect a *lot* of people have links to the  
python-dev archive. Hope you have a backup (or can find caches on  
google or archive.org or something).

James


From barry at python.org  Tue Jan 19 17:44:21 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:44:21 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119114421.3b96fbd4@freewill>

On Jan 19, 2010, at 11:24 AM, James Y Knight wrote:

>No doubt, you've now broken every link anyone had ever made into the  
>python-dev archives, because now all the article numbers are  
>different. BTDT...unfortunately... Pipermail really is quite crappy,  
>sigh.

I've been trying for 10+ years to get folks interested in helping me fix this
(and a few other warts) in Pipermail, to not much success. ;/

>Anyhow, when I did that, I went back to a backup to get the original  
>article numbers, and edited the mbox file escaping From lines or  
>adding additional empty messages until the newly regenerated article  
>numbers matched the originals. I'd highly recommend going through that  
>painful process, since I suspect a *lot* of people have links to the  
>python-dev archive. Hope you have a backup (or can find caches on  
>google or archive.org or something).

bin/cleanarch uses a set of heuristics to find unescaped From lines and fix
them.  It's generally pretty good, but it certain can change message numbers
(and sadly, their urls).

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/e32aa882/attachment-0002.pgp>

From rdmurray at bitdance.com  Tue Jan 19 18:48:41 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Jan 2010 12:48:41 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119174841.9BC3B16A53@kimball.webabinitio.net>

On Tue, 19 Jan 2010 11:24:57 -0500, James Y Knight <foom at fuhm.net> wrote:
> 
> On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:
> 
> > On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
> >
> >> When I look at the mailing list archive for python-dev, I see some  
> >> odd stuff at the bottom of the page:
> >>
> >> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
> >>
> >> Anyone know what's happened?
> >
> > WTF?  I think the archives were recently regenerated, so there's  
> > probably a fubar there.  CC'ing the postmasters.
> 
> That happens if messages had unescaped "From" lines in the middle of  
> them.
> 
> No doubt, you've now broken every link anyone had ever made into the  
> python-dev archives, because now all the article numbers are  
> different. BTDT...unfortunately... Pipermail really is quite crappy,  
> sigh.
> 
> Anyhow, when I did that, I went back to a backup to get the original  
> article numbers, and edited the mbox file escaping From lines or  
> adding additional empty messages until the newly regenerated article  
> numbers matched the originals. I'd highly recommend going through that  
> painful process, since I suspect a *lot* of people have links to the  
> python-dev archive. Hope you have a backup (or can find caches on  
> google or archive.org or something).

The Python issue tracker does, for one.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From ncoghlan at gmail.com  Tue Jan 19 22:18:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 07:18:52 +1000
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119174841.9BC3B16A53@kimball.webabinitio.net>
References: <loom.20100119T164757-651@post.gmane.org>	<20100119110748.69bc564a@freewill>	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
	<20100119174841.9BC3B16A53@kimball.webabinitio.net>
Message-ID: <4B5621BC.4070608@gmail.com>

R. David Murray wrote:
> The Python issue tracker does, for one.

And all the PEPs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From david.lyon at pythontest.org  Wed Jan 20 00:16:44 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 10:16:44 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119081639.5d431ed9@freewill>
References: <20100119081639.5d431ed9@freewill>
Message-ID: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>


Hi Barry,

That looks very interesting...

So does that mean we could update the stdlib for a given
python version using this ?

David

> I've just updated the Launchpad mirrors for the 4 active Python branches,
> trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar
> branches
> on code.python.org but it's probably been 7 months or so since those were
> regularly updated.  Now the Launchpad branches sync against the read-only
> Subversion branches at http://svn.python.org, so they should remain
> up-to-date
> (within the re-sync timeframe of about 4 hours).
>
> This means you can once again use Bazaar to get local branches of Python,
> and
> you can of course push your own branches to Launchpad.  I believe you can
> even
> use the bzr-svn plugin to commit changes back to the Subversion master,
> though
> I have not yet tried this.
>
> To get a local branch, just do any of the following:
>
>     % bzr branch lp:python (for trunk)
>     % bzr branch lp:python/2.6
>     % bzr branch lp:python/py3k
>     % bzr branch lp:python/3.1
>
> (It's fairly easy to create new mirrors for other Subversion branches,
> e.g. Python 2.5; just drop me an email if you want them.)
>
> If you're going to create a lot of branches you probably want to put them
> in a
> shared repository.  E.g.
>
>     % bzr init-repo pythonbzr
>     % cd pythonbzr
>     % bzr branch lp:python/py3k
>
> Bazaar 2.0 or better is recommended.  For me, it took about 5m to check
> the
> first branch out from Launchpad, and then about 30s or so for each
> subsequent
> branch.
>
> Enjoy,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/david.lyon%40pythontest.org
>




From barry at python.org  Wed Jan 20 00:54:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 18:54:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119185439.299660c7@freewill>

On Jan 20, 2010, at 10:16 AM, David Lyon wrote:

>Hi Barry,
>
>That looks very interesting...

Hi David,

>So does that mean we could update the stdlib for a given
>python version using this ?

In a sense, yes (if I understand your question correctly).

You can use Bazaar to branch any of the 4 Python series and use all the modern
DVCS goodness to develop your updates.  You can share your branches with
others via e.g. Launchpad and even request reviews (called "merge proposals")
to get feedback from others.

The one thing I am unsure about, mostly because I have not tried it, is
whether your Bazaar branch can be used to commit directly back to the Python
Subversion master branches.  I /think/ the answer is yes, assuming of course
that you have permission to do so, and that you have a modern version of
Bazaar and the bzr-svn plugin.

It might even be possible to commit your Bazaar branch to a local Subversion
branch, and then commit the latter to get it pushed up to svn.python.org.

At worst, you would use Bazaar's features to get your patch into a state you
and your reviewers are happy with, then you would generate a diff for
application to your copy of the svn.python.org branch.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/37753b1a/attachment-0002.pgp>

From david.lyon at pythontest.org  Wed Jan 20 01:51:23 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 11:51:23 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119185439.299660c7@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
Message-ID: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>

> On Jan 20, 2010, at 10:16 AM, Barry wrote:

>> So does that mean we could update the stdlib for a given
>> python version using this ?
>
> In a sense, yes (if I understand your question correctly).

Yeah, it just needs an implementation.

> The one thing I am unsure about, mostly because I have not tried it, is
> whether your Bazaar branch can be used to commit directly back to the
> Python Subversion master branches.  I /think/ the answer is yes,
> assuming of course that you have permission to do so...

Well I'm too Senior and my stuff is too forward looking to qualify
for that just yet.

I'd be happy to see bzr and mercurial and git all made it together
into the stdlib for python 3. That would give a superb updating
mechanism for python that would propel python well beyond
the dinosaur badlands of CPAN and other languages.

I was actually reading from
(http://en.wikipedia.org/wiki/Python_%28programming_language%29):

"Rather than requiring all desired functionality to be built into the
language's core, Python was designed to be highly extensible. .. .. This
design of a small core language with a large standard library and an
easily extensible interpreter was intended by Van Rossum from the very
start because of his frustrations with ABC (which espoused the opposite
mindset).[5]"

To me, the source code control systems seem to be fully in tune
with the original design of python. That is, to be able to
easily pull external libraries in.

I think what has changed is that the mechanisms now (the SCMs)
are way more highly developed than before. Apart from that
though, after reading the full wikipedia article I'm left
with the distinct impression that things are still pretty
much the same (in that python design philosophy is advanced),
just that the landscape (of external C libraries) has changed.

Now all the libraries are external (on the internet) and
all externally managed.

So with just a tiny amount of work, imho we could pull
it all together to bring python 3 *back* to being that
cool tool that it once was (not saying it isn't now).

Were you offering me an experimental branch somewhere
for python 3 SCM integration ?

David







From jnoller at gmail.com  Wed Jan 20 02:09:15 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 20:09:15 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>

On Tue, Jan 19, 2010 at 7:51 PM, David Lyon <david.lyon at pythontest.org> wrote:
>> On Jan 20, 2010, at 10:16 AM, Barry wrote:
>
>>> So does that mean we could update the stdlib for a given
>>> python version using this ?
>>
>> In a sense, yes (if I understand your question correctly).
>
> Yeah, it just needs an implementation.
>
>> The one thing I am unsure about, mostly because I have not tried it, is
>> whether your Bazaar branch can be used to commit directly back to the
>> Python Subversion master branches. ?I /think/ the answer is yes,
>> assuming of course that you have permission to do so...
>
> Well I'm too Senior and my stuff is too forward looking to qualify
> for that just yet.
>
> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

i sincerely doubt that a source control system will be included in the
standard library in the future. Especially 3. A SCM is not a "package
management system".

Barry was talking about mirrors of the python code. It is true a
"package manager" could be developed based on a SCM, however you need
to implement this far away from the stdlib and get traction with it
within the community long before inclusion would be considered.

The decision to move python's source control from SVN to mercurial was
controversial enough; including 3 or more scm libraries into core
would be an intractable uphill mountain of bike sheds.

> So with just a tiny amount of work, imho we could pull
> it all together to bring python 3 *back* to being that
> cool tool that it once was (not saying it isn't now).

Python 3 is still modularized, still has a standard library, etc. If
you're really interested in helping with the standard library, get on
stdlib-sig, and get ready to write code and PEPs.

> Were you offering me an experimental branch somewhere
> for python 3 SCM integration ?

Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

jesse


From barry at python.org  Wed Jan 20 04:32:30 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 22:32:30 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <20100119223230.4c4a7ed5@freewill>

On Jan 19, 2010, at 08:09 PM, Jesse Noller wrote:

>The decision to move python's source control from SVN to mercurial was
>controversial enough; including 3 or more scm libraries into core
>would be an intractable uphill mountain of bike sheds.

I'd be surprised if any of the big 3 DVCS developers would actually /want/
their stuff in the stdlib.  Being in the stdlib has its advantages and
disadvantages.  I think for rapidly developing technology, the latter can
actually outweigh the former.

(Besides, git in the stdlib doesn't make much sense :).

>Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

Bingo.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/9ef0ed2b/attachment-0002.pgp>

From david.lyon at pythontest.org  Wed Jan 20 04:43:12 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 14:43:12 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>


> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

> Python 3 is still modularized, still has a standard library, etc. If
> you're really interested in helping with the standard library, get on
> stdlib-sig, and get ready to write code and PEPs.

Thank you for your direction to move these items forward to PEPs
and Code.

> i sincerely doubt that a source control system will be included in the
> standard library in the future. Especially 3.

Yeah and who twenty years ago thought you would get a 1GB memory
card for $3 when all we had was 10Meg hard disks and they were
the full 8" platter.

> A SCM is not a "package management system".

Exactly. It almost makes the need for a "package management system"
pretty much obsolete if you can update your code directly from
the developers sources.

That's what all these SCMs provide. Plus it's addictive. It's
hard to go back to 'package' style technology once you have
all your code on an SCM based feed.

> Barry was talking about mirrors of the python code. It is true a
> "package manager" could be developed based on a SCM, however you need
> to implement this far away from the stdlib and get traction with it
> within the community long before inclusion would be considered.

I think I'll have better chances with PEPs.

Being honest, if wonderful libraries like Sphinx and Mercurial
and Git and BZR can't make it into the stdlib, then there is
no hope for even newer code to get in there.

Plus, promoting all sorts of new and fangled tools however
good they may or may not be just confuses users and ends
up being a waste of time imho. It isn't good management
of volounteers time and effort.

If you could imagine disaster relief coordinated this
way, it would just be a disaster in itself.

That's why it has taken some 5 years to get PEP-345 done.

> The decision to move python's source control from SVN to mercurial was
> controversial enough; including 3 or more scm libraries into core
> would be an intractable uphill mountain of bike sheds.

Not at all.

It would be a very fair thing to do. Not to mention being
great for users.

> Barry made bzr mirrors of the python svn tree. Not a python with bzr
> included.

I can't resist asking for that again.. I heard it only in Monty
speek. Did you just say ?:

 "Barry made a bizarre mirror of the python suvern tree. Not a
  python with a buzzer included."

Anyway.. Maybe I do get what your talking about. Even if you do
talk with a strange accent. :-)

David






From jnoller at gmail.com  Wed Jan 20 05:10:07 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 23:10:07 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001192010v66b728dbxa7ad1ed1fc1df15f@mail.gmail.com>

On Tue, Jan 19, 2010 at 10:43 PM, David Lyon <david.lyon at pythontest.org> wrote:
[snip]
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.

Did you ever stop to think that some package authors do not want their
code in the standard library? That throwing random shiny things in
there just makes it a junk drawer?

Besides: Show us the PEP to include Sphinx, and it's dependencies in
the standard lib, with Georg's signature at the bottom.

The authors of modules have to want things to be in there, they have
to be best of breed, tested or common-enough patterns to warrant a
slot. We have things in the standard lib which still need more TLC and
maintenance, or they need to be shunted into space.

Adding random things, which may or may not help packaging, and
installing things from random places in someone source repository
won't make things better.

> Plus, promoting all sorts of new and fangled tools however
> good they may or may not be just confuses users and ends
> up being a waste of time imho. It isn't good management
> of volounteers time and effort.
>
> If you could imagine disaster relief coordinated this
> way, it would just be a disaster in itself.
>
> That's why it has taken some 5 years to get PEP-345 done.

I'm going to assume that you're trolling now, or intentionally
misrepresenting facts. Maybe a little of both. A PEP, and an
implementation and the ability to rationally debate, discuss and
defend your proposal is what is needed to enact changes on policies,
python-core or the standard library.

>> The decision to move python's source control from SVN to mercurial was
>> controversial enough; including 3 or more scm libraries into core
>> would be an intractable uphill mountain of bike sheds.
>
> Not at all.
>
> It would be a very fair thing to do. Not to mention being
> great for users.

There should be one-- and preferably only one --obvious way to do it.

> Anyway.. Maybe I do get what your talking about. Even if you do
> talk with a strange accent. :-)

My sense of humor has been disabled by repeated stunning at your
hands. I admire your enthusiasm, even if I do think some of it is
misplaced, or at guided into the proper channels at very least.

Please, you seem to have the time and willingness to help, please go
about this the right way. Discuss things on the proper lists, make
concrete proposals. If you have have standard lib changes, discuss
them on stdlib-sig, if you have ideas about python-the-language, or
the interpreter, etc - please discuss it on python-ideas.

jesse


From ben+python at benfinney.id.au  Wed Jan 20 05:16:25 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 15:16:25 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87wrzdfm1y.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
> and BZR can't make it into the stdlib, then there is no hope for even
> newer code to get in there.

Those are applications, not libraries. Applications don't belong in the
standard library.

-- 
 \     ?If you pick up a starving dog and make him prosperous, he will |
  `\      not bite you. This is the principal difference between a dog |
_o__)                    and a man.? ?Mark Twain, _Pudd'n'head Wilson_ |
Ben Finney



From brian.curtin at gmail.com  Wed Jan 20 05:19:47 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 19 Jan 2010 22:19:47 -0600
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <cf9f31f21001192019p25665318t3d99caf3db27198e@mail.gmail.com>

On Tue, Jan 19, 2010 at 21:43, David Lyon <david.lyon at pythontest.org> wrote:

>
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
>

I'm not entirely sure I see why the inclusion of a SCM into the stdlib is
necessary.

Just because pieces of software are mature and proven in their fields
doesn't mean we should add them, or that them *not* being in the stdlib
should be a basis for other projects making it in.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/3a2d80f8/attachment-0002.htm>

From barry at python.org  Wed Jan 20 05:26:44 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:26:44 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119232644.7fa25691@freewill>

On Jan 20, 2010, at 02:43 PM, David Lyon wrote:

>> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

>> A SCM is not a "package management system".
>
>Exactly. It almost makes the need for a "package management system"
>pretty much obsolete if you can update your code directly from
>the developers sources.
>
>That's what all these SCMs provide. Plus it's addictive. It's
>hard to go back to 'package' style technology once you have
>all your code on an SCM based feed.

Well...  I'm not so sure.  A package management system like apt does a /ton/
of additional bookkeeping and work to ensure a robust, highly consistent,
functioning system.  And while both Python and most Linux distributions have
their own notion of "package management", they don't always play nicely
together.  Tarek and the distutils-sig's work is trying to make the world a
better place by bridging this gap better, and there is code out there that
makes it easier to say import a Python package from the Cheeseshop and
.deb-ify it for use on Debian and Ubuntu.

There's also work being done in Launchpad that will allow you to
"build-from-branch" so that in a sense you could let a build farm take your
Bazaar branches and automatically build the packages from them.

I've strayed off-topic I suppose, but I see SCMs and package managers as
complementary technologies that help with important parts of the process of
delivering software to end-users, but I don't quite see how one can make the
other obsolete.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/4be8f756/attachment-0002.pgp>

From david.lyon at pythontest.org  Wed Jan 20 05:29:34 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 15:29:34 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>

> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>
> I'd be surprised if any of the big 3 DVCS developers would actually /want/
> their stuff in the stdlib.

If they ask, they'll get told they're motorbike-shedding. "It's better
if their users ask". So here I am as a user doing things the 'right'
way.

> Being in the stdlib has its advantages and
> disadvantages.  I think for rapidly developing technology, the
> latter can actually outweigh the former.

If it's about being able to do updates, then I think this
resolves an old and circular argument. As the SCM implementation
would, one would expect, to be able to update itself.

Side benefits are that it can update everything else along
with it at the same time. User Apps, Packages, whatever.

It's even better having SCM in an Industrial/Scientific
environment. Here's an example:

 - a machine breaks..  (I mean the software for/in it)

 - you fix the code, maybe on the spot

 - you commit and push back to the repository

 - your code gets checked in and run through the testbot
   and then you get blamed and have to do the whole thing
   again properly with a test case. Oh well..

Well anyway, whatever you guys might say, that's a whole
lot more efficient than running back to the development
machine and going through some obscure build and test
and publish process to do a fix on a production machine.

Point : The fact that SCMs are two way is great in
        a production environment. No packaging solution
        can come close.

So why not have python SCMs included as batteries in python..

All these arguments I can take off to the stdlib list when I get
the chance..

David




From barry at python.org  Wed Jan 20 05:50:36 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:50:36 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119235036.57f6e867@freewill>

Okay, last follow up on this and then I'm going to bed. :)

On Jan 20, 2010, at 03:29 PM, David Lyon wrote:

>> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>>
>> I'd be surprised if any of the big 3 DVCS developers would actually /want/
>> their stuff in the stdlib.
>
>If they ask, they'll get told they're motorbike-shedding. "It's better
>if their users ask". So here I am as a user doing things the 'right'
>way.

Actually, you're not.  It's not up to the Python community to initiate this.
If you really want this, you should engage with the relevant DVCS communities
and push them to request it.

>Side benefits are that it can update everything else along
>with it at the same time. User Apps, Packages, whatever.

I get that.  Heck, I still run one Gentoo server which I think is as close to
the edge you're describing as I'm comfortable with.  It's all great until the
wheels come off and then it can take *days* to get a functioning system
again.

The big difference is that I rely on my DVCS to keep one small thing, or a few
variants of the same thing, all sane.  But I rely on my distribution vendor to
keep a thousand complex, interdependent, interacting, sometimes conflicting
things sane and working.

>Point : The fact that SCMs are two way is great in
>        a production environment. No packaging solution
>        can come close.

Try talking with some hard-core operations guys, the folks with the keys to
the data centers, who work tireless, insanely hours keeping incredibly complex
systems running with very little downtime.  I think you'd get a different
perspective to put it mildly. :)

to-sleep-perchance-to-dream-ly y'rs,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/0def1cc9/attachment-0002.pgp>

From david.lyon at pythontest.org  Wed Jan 20 06:10:15 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 16:10:15 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <87wrzdfm1y.fsf@benfinney.id.au>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
Message-ID: <1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>

> "David Lyon" <david.lyon at pythontest.org> writes:
>
>> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
>> and BZR can't make it into the stdlib, then there is no hope for even
>> newer code to get in there.
>
> Those are applications, not libraries. Applications don't belong in the
> standard library.

Haha funny..

Well using that logic, distutils is an application..

Are you saying that distutils should be removed? That
is most certainly an application.

Lets not get too pedantic here. Mercurial and bzr have a built
in API that can be called in a library like way. It's true they
also have a command line interface in the same way that distutils
does.

I'm not saying anything negative about distutils. Given that
Tarek has an upcoming Pycon presentation where the program
talks about a distutils revamp.

I'm hoping that he can find some young 20 yr olds and
put a cool web interface on that thing. Given that there
are empty sprints at pycon. It couldn't hurt to throw
that challenge out. Anyway, we'll see..


David




From ben+python at benfinney.id.au  Wed Jan 20 06:32:10 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:32:10 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
	<20100119235036.57f6e867@freewill>
Message-ID: <87iqaxfijp.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> On Jan 20, 2010, at 03:29 PM, David Lyon wrote:
> >So here I am as a user doing things the 'right' way.
>
> Actually, you're not. It's not up to the Python community to initiate
> this. If you really want this, you should engage with the relevant
> DVCS communities and push them to request it.

Where ?push? must be strictly limited by a continual awareness that the
whole idea could just be bad.

If you find yourself in a tiny minority pushing for a change, it *could*
be that you have a great idea and the vast majority don't realise it
yet. But you must be realistic about the likelihood that the change is a
very *bad* idea, and frequently evaluate it for signs of that.

-- 
 \     ?I used to think that the brain was the most wonderful organ in |
  `\   my body. Then I realized who was telling me this.? ?Emo Philips |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Wed Jan 20 06:34:07 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:34:07 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
	<1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87eillfigg.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Well using that logic, distutils is an application..

Distutils is an application, the function of which is essential to
allowing sane development of Python packages. It's a special case. We
need to strictly limit the number of special cases, not gleefully add to
them.

-- 
 \        ?I'd take the awe of understanding over the awe of ignorance |
  `\                                          any day.? ?Douglas Adams |
_o__)                                                                  |
Ben Finney



From matthieu.brucher at gmail.com  Wed Jan 20 07:49:36 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 07:49:36 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>

> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

I think there are several points that make them not includable in Python:
- git is not written in Python
- bzr and mercurial have a life cycle much shorter than Python's, it's
the same issue than with other libraries where another community
develops them.

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From david.lyon at pythontest.org  Wed Jan 20 08:20:02 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 18:20:02 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
Message-ID: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>


Matthieu,

>> I'd be happy to see bzr and mercurial and git all made it together
>> into the stdlib for python 3. That would give a superb updating
>> mechanism for python that would propel python well beyond
>> the dinosaur badlands of CPAN and other languages.
>
> I think there are several points that make them not includable in Python:
> - git is not written in Python
> - bzr and mercurial have a life cycle much shorter than Python's, it's
> the same issue than with other libraries where another community
> develops them.

That's only two points. :-)

On 1; If that's true, I won't mention git again.

On 2; Who knows what their life cycle is. CVS is pretty much
      dead, and svn looks like it is on the way out.
      I can't think of how anything could be better than
      mercurial or bzr but I know I will be proved wrong.

At the end of the day, we are making a decision about whether
the language is 'set-in-stone' or whether it is still
evolving.

To me, Python 1.x had it's own distinct "era", as has
Python 2.x

Hoping that the Python 3 era can be a little more flexible
and perphaps "cleaner" than the 2.x era is all that I am
thinking here.

Have a nice day

David





From matthieu.brucher at gmail.com  Wed Jan 20 09:05:19 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 09:05:19 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001200005j6344672fo99826cc2e8648141@mail.gmail.com>

> That's only two points. :-)

In French, we say that several starts with 2 ;)

> On 1; If that's true, I won't mention git again.

I tis, you can check on the git repository (it's a mix of C, perl,
shell scripts, Python, ...)

> On 2; Who knows what their life cycle is.

You can check on their websites, their cycles are far shorter than
Python minor releases (several months vs several years).

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From abpillai at gmail.com  Wed Jan 20 10:36:53 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Wed, 20 Jan 2010 15:06:53 +0530
Subject: [Python-Dev] E3 BEFEHLE
In-Reply-To: <e9764b731001162012y519de6dbk74aab0930e3cf18c@mail.gmail.com>
References: <hit9gr$hal$1@ger.gmane.org>
	<b8e622741001161932n15a0e0d6i69ec06c1f820e6b@mail.gmail.com>
	<e9764b731001162012y519de6dbk74aab0930e3cf18c@mail.gmail.com>
Message-ID: <8548c5f31001200136r54bc9e2aw49485280864ecb2d@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:42 AM, Dj Gilcrease <digitalxero at gmail.com> wrote:

> 2010/1/16 Jack Diederich <jackdied at gmail.com>:
> > Good lord, did this make it past other people's spam filters too?  I
> > especially liked the reference to "REGION -2,0 ; Rlyeh".  Ph'nglui
> > mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn to you too sir.
>
> Ya made it past mine too, it looks like a debug dump of a macro for a
> some German game based either LOTR or Cthulhu
>

I initially thought it was  a Python disassembler trace of some step
of operations which failed, converted to German.

In fact, I was looking for a question at the end regarding REPL.
How very optimistic...


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100120/5dd68498/attachment-0002.htm>

From stephen at xemacs.org  Wed Jan 20 11:22:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 20 Jan 2010 19:22:55 +0900
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <87aaw9gjnk.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > (Besides, git in the stdlib doesn't make much sense :).

"Dulwich."


From solipsis at pitrou.net  Wed Jan 20 12:28:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Jan 2010 11:28:16 +0000 (UTC)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100120T122742-162@post.gmane.org>

David Lyon <david.lyon <at> pythontest.org> writes:
> 
> I think I'll have better chances with PEPs.
> 
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
> [snip]

This is python-ideas material, can you take it there? Thank you.

Antoine.




From ncoghlan at gmail.com  Wed Jan 20 13:16:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 22:16:34 +1000
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>	<20100119185439.299660c7@freewill>	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B56F422.5010607@gmail.com>

David Lyon wrote:
> On 2; Who knows what their life cycle is. CVS is pretty much
>       dead, and svn looks like it is on the way out.
>       I can't think of how anything could be better than
>       mercurial or bzr but I know I will be proved wrong.

I believe you misunderstood what Matthieu meant by life cycle there:
think "release cycle". If a project pushes out new releases
significantly more often than every 18-24 months (as is currently true
for all of the major SCM tools), then that fact alone makes it a very
bad fit for the Python standard library.

And centralised source control will be going strong for years. The DVCS
approach may be great for the open source world, but the gains are far
more limited in a closed source shop (especially a group writing
internal corporate applications which doesn't need to keep many, if any,
maintenance branches going).

If we weren't dealing with 4 active branches, the DVCS discussion would
have got a lot less traction with the core developers - aside from
better handling of multiple lines of development, most of the benefits
of the switch to a DVCS accrue to people without commit access to the
SVN repository.

Anyway, we've wandered far afield from legit python-dev topics now. Any
further ideas about super_mega_easy_install functionality that can pull
code from source control systems and build it rather than requiring
prebuild source tarballs should be directed to python-ideas (they
probably need to bake more even before they make an appearance on
distutils-sig).

Cheers,
Nick.

P.S. As Jesse said... your enthusiasm is great, but please don't assume
that some inherent conservatism on the part of other developers is
automatically evil or the result of a failure to see your point. A lot
of people around the world rely on our stuff every day. We owe it to
them to be measured in our actions and to put serious thought into any
major changes or additions we make to the language and the standard
library. For the current stage of its development, Python 3 is in a good
place from our point of view - its major carrot has really always been
the better Unicode support it offers, and the ever-increasing
globalisation of the web will create more and more pressure pushing
developers in that direction as the years go by. Sure, Python 3 cleans
up assorted other things as well, but the change to the text processing
model is the big one that is fundamentally incompatible with the
architecture of the 2.x series. Compared to that change, everything else
is just tinkering.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ziade.tarek at gmail.com  Fri Jan  1 16:07:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:07:20 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04D81D51@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<94bdd2610912141458m52da21ear4970506a37214e6@mail.gmail.com>
	<4dab5f760912230700l38a597f7l855bb2304025d6d5@mail.gmail.com>
	<F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
Message-ID: <94bdd2611001010707h4043ff64x7476049e435852b@mail.gmail.com>

2009/12/23 Glyph Lefkowitz <glyph at twistedmatrix.com>:
[..]
> 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages".
>
> The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. ?In other words it's got "bin", "lib", "share", "etc", etc.. ?~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation.

Right, good idea

Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Fri Jan  1 16:32:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:32:27 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
Message-ID: <94bdd2611001010732o38a563bcu6d0cc9122ed5c88f@mail.gmail.com>

On Sat, Dec 12, 2009 at 9:02 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> A while ago I've proposed to refactor the APIs that provides access to
> the installation paths and configuration variables at runtime into a
> single module called "sysconfig", and make it easier for all
> implementations to work with them.

I've applied the suggestions made in this thread.

If no one objects, here's what I am going to do now:

I'll merge this change in the trunk, (I'll drop the branch changesets
in favor of one single, clean changeset) and start to work on the
corresponding doc, so more people will be able to give some feedback
on the APIs once the second 2.7 alpha is out.

Then, in a second step, I'll work on the removal of the Makefile and
pyconfig.h dependencies by adding a _synconfig.py file as suggested
earlier, that will be created during the build process.

Regards,
Tarek


From status at bugs.python.org  Fri Jan  1 18:07:11 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jan 2010 18:07:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100101170711.A5AAF78654@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/25/09 - 01/01/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2537 open (+22) / 16902 closed (+19) / 19439 total (+41)

Open issues with patches:  1016

Average duration of open issues: 705 days.
Median duration of open issues: 462 days.

Open Issues Breakdown
   open  2504 (+22)
pending    32 ( +0)

Issues Created Or Reopened (42)
_______________________________

doc: Code is not always colored                                  12/30/09
CLOSED http://bugs.python.org/issue7487    reopened flox                          
                                                                               

documention buglet for PyBuffer                                  12/26/09
CLOSED http://bugs.python.org/issue7577    created  ronaldoussoren                
       easy                                                                    

Behavior of operations on a closed file object is not documented 12/26/09
       http://bugs.python.org/issue7578    created  blep                          
                                                                               

Patch to add docstrings to msvcrt                                12/26/09
       http://bugs.python.org/issue7579    created  brian.curtin                  
       patch                                                                   

distutils makefile from python.org installer doesn't work on OS  12/27/09
       http://bugs.python.org/issue7580    created  bskaplan                      
                                                                               

incomplete doc of zlib                                           12/27/09
CLOSED http://bugs.python.org/issue7581    created  coolwanglu                    
                                                                               

[patch] diff.py to use iso timestamp                             12/27/09
       http://bugs.python.org/issue7582    created  techtonik                     
       patch                                                                   

doctest should normalize tabs when comparing output              12/27/09
       http://bugs.python.org/issue7583    created  techtonik                     
                                                                               

datetime.rfcformat() for Date and Time on the Internet           12/27/09
       http://bugs.python.org/issue7584    created  techtonik                     
                                                                               

[patch] difflib should separate filename from timestamp with tab 12/27/09
       http://bugs.python.org/issue7585    created  techtonik                     
       patch                                                                   

Typo in collections documentation                                12/28/09
CLOSED http://bugs.python.org/issue7586    created  nneonneo                      
                                                                               

Python 3.1.1 source build issues                                 12/28/09
CLOSED http://bugs.python.org/issue7587    created  sharmaag                      
                                                                               

unittest.TestCase.shortDescription isn't short anymore           12/28/09
       http://bugs.python.org/issue7588    created  exarkun                       
                                                                               

setup.py shouldn't try to build nis module when nis headers aren 12/28/09
CLOSED http://bugs.python.org/issue7589    created  Arfrever                      
       patch                                                                   

'exceptions' module mentioned in documentation                   12/28/09
CLOSED http://bugs.python.org/issue7590    created  July                          
                                                                               

test_get_platform fails on 3.1                                   12/28/09
       http://bugs.python.org/issue7591    created  pitrou                        
       buildbot                                                                

ssl module documentation: SSLSocket.unwrap description shown twi 12/28/09
       http://bugs.python.org/issue7592    created  mnewman                       
                                                                               

Computed-goto patch for RE engine                                12/29/09
       http://bugs.python.org/issue7593    created  akuchling                     
       patch, needs review                                                     

shlex refactoring                                                12/29/09
       http://bugs.python.org/issue7594    created  ferringb                      
       patch, needs review                                                     

Doc typo for select.kevent()                                     12/29/09
CLOSED http://bugs.python.org/issue7595    created  whit537                       
                                                                               

test_logging sometimes fails                                     12/29/09
CLOSED http://bugs.python.org/issue7596    created  pitrou                        
                                                                               

curses.use_env implementation error                              12/30/09
       http://bugs.python.org/issue7597    created  kanru                         
       patch                                                                   

Cannot install, problems with assembly?                          12/30/09
CLOSED http://bugs.python.org/issue7598    created  sh.00                         
                                                                               

(c)ElementTree can produce invalid XML                           12/30/09
       http://bugs.python.org/issue7599    created  jwilk                         
                                                                               

interpreter crashes before start                                 12/30/09
CLOSED http://bugs.python.org/issue7600    created  mhh91                         
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc         12/30/09
CLOSED http://bugs.python.org/issue7601    created  sadhak                        
                                                                               

Doc: make clean and make update do not delete or update Doc/tool 12/30/09
CLOSED http://bugs.python.org/issue7602    created  flox                          
                                                                               

There is no 'seq' type                                           12/30/09
CLOSED http://bugs.python.org/issue7603    created  tom66                         
                                                                               

delattr __slots__ inconsistancy                                  12/30/09
CLOSED http://bugs.python.org/issue7604    created  ferringb                      
                                                                               

test_cmd_line fails with non-ascii path                          12/30/09
       http://bugs.python.org/issue7605    created  pitrou                        
                                                                               

test_xmlrpc fails with non-ascii path                            12/30/09
       http://bugs.python.org/issue7606    created  pitrou                        
                                                                               

stringlib fastsearch could be improved on 64-bit builds          12/30/09
       http://bugs.python.org/issue7607    created  pitrou                        
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.             12/30/09
CLOSED http://bugs.python.org/issue7608    created  alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                   12/31/09
CLOSED http://bugs.python.org/issue7609    created  Arfrever                      
       patch                                                                   

Cannot use both read and readline method in same ZipExtFile obje 12/31/09
       http://bugs.python.org/issue7610    created  lucifer                       
                                                                               

shlex not posix compliant when parsing "foo#bar"                 12/31/09
       http://bugs.python.org/issue7611    created  jjdmol2                       
                                                                               

Fix "pass and object" typo in Library Reference / Built-in Types 12/31/09
CLOSED http://bugs.python.org/issue7612    created  ygale                         
       patch                                                                   

[cppcheck] found a regression : Invalid number of character ((). 12/31/09
CLOSED http://bugs.python.org/issue7613    created  ettl.martin                   
       patch                                                                   

Python 2.6.4 segfaults                                           12/31/09
CLOSED http://bugs.python.org/issue7614    created  ttsiod                        
                                                                               

unicode_escape codec does not escape quotes                      01/01/10
       http://bugs.python.org/issue7615    created  rhansen                       
       patch                                                                   

test_memoryview test_setitem_writable failures with Intel ICC    01/01/10
       http://bugs.python.org/issue7616    created  ivank                         
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option 01/01/10
       http://bugs.python.org/issue7617    created  Arfrever                      
       patch                                                                   



Issues Now Closed (46)
______________________

True division of integers could be more accurate                  715 days
       http://bugs.python.org/issue1811    mark.dickinson                
       patch                                                                   

API to clear most free lists                                      690 days
       http://bugs.python.org/issue2019    amaury.forgeotdarc            
       patch                                                                   

unit test UnicodeWarning                                          687 days
       http://bugs.python.org/issue2100    ezio.melotti                  
                                                                               

test_disutils fails                                               568 days
       http://bugs.python.org/issue3054    ronaldoussoren                
       patch                                                                   

test_formatdate_usegmt fails on non-englisch locale               451 days
       http://bugs.python.org/issue4046    r.david.murray                
                                                                               

"??" in u"foo" raises a misleading error                          410 days
       http://bugs.python.org/issue4328    r.david.murray                
                                                                               

zipfile - add __exit__ attribute to make ZipFile object compatib  287 days
       http://bugs.python.org/issue5511    ezio.melotti                  
       patch                                                                   

test_urllib2 fails - urlopen error file not on local host         271 days
       http://bugs.python.org/issue5625    orsenthil                     
                                                                               

Improve --with-dbmliborder option                                 170 days
       http://bugs.python.org/issue6491    benjamin.peterson             
       patch                                                                   

Move the special-case for integer objects out of PyBytes_FromObj  141 days
       http://bugs.python.org/issue6687    alexandre.vassalotti          
       patch, 26backport                                                       

setup.py fails to find headers of system libffi                   104 days
       http://bugs.python.org/issue6943    benjamin.peterson             
       patch, easy                                                             

The replacement suggested for callable(x) in py3k is not	equival   92 days
       http://bugs.python.org/issue7006    benjamin.peterson             
       patch                                                                   

C/API - Document exceptions                                        89 days
       http://bugs.python.org/issue7033    lekma                         
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi   35 days
       http://bugs.python.org/issue7381    georg.brandl                  
       patch                                                                   

optparse Documentation References Example Files that do not Exis   30 days
       http://bugs.python.org/issue7404    georg.brandl                  
       patch                                                                   

datetime.datetime.isoformat truncation problem                     29 days
       http://bugs.python.org/issue7413    amaury.forgeotdarc            
       patch, needs review                                                     

doc: Code is not always colored                                     0 days
       http://bugs.python.org/issue7487    georg.brandl                  
                                                                               

float('nan')**2 != nan. float('nan')**2 error 33 on windows        13 days
       http://bugs.python.org/issue7534    mark.dickinson                
       patch                                                                   

If a generator raises TypeError when being unpacked, an unrelate   10 days
       http://bugs.python.org/issue7548    amaury.forgeotdarc            
                                                                               

ctypes doc improvement: c_char_p                                    6 days
       http://bugs.python.org/issue7569    georg.brandl                  
                                                                               

Strange issue : cursor.commit() with sqlite                         5 days
       http://bugs.python.org/issue7572    ghaering                      
                                                                               

tes_math fails Mac OS X 10.4 due to OverflowError in test_mtestf    5 days
       http://bugs.python.org/issue7575    mark.dickinson                
       patch                                                                   

documention buglet for PyBuffer                                     2 days
       http://bugs.python.org/issue7577    georg.brandl                  
       easy                                                                    

incomplete doc of zlib                                              0 days
       http://bugs.python.org/issue7581    amaury.forgeotdarc            
                                                                               

Typo in collections documentation                                   0 days
       http://bugs.python.org/issue7586    georg.brandl                  
                                                                               

Python 3.1.1 source build issues                                    0 days
       http://bugs.python.org/issue7587    amaury.forgeotdarc            
                                                                               

setup.py shouldn't try to build nis module when nis headers aren    1 days
       http://bugs.python.org/issue7589    benjamin.peterson             
       patch                                                                   

'exceptions' module mentioned in documentation                      1 days
       http://bugs.python.org/issue7590    georg.brandl                  
                                                                               

Doc typo for select.kevent()                                        0 days
       http://bugs.python.org/issue7595    georg.brandl                  
                                                                               

test_logging sometimes fails                                        1 days
       http://bugs.python.org/issue7596    ezio.melotti                  
                                                                               

Cannot install, problems with assembly?                             0 days
       http://bugs.python.org/issue7598    ezio.melotti                  
                                                                               

interpreter crashes before start                                    0 days
       http://bugs.python.org/issue7600    r.david.murray                
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            1 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

Doc: make clean and make update do not delete or update Doc/tool    0 days
       http://bugs.python.org/issue7602    georg.brandl                  
                                                                               

There is no 'seq' type                                              0 days
       http://bugs.python.org/issue7603    benjamin.peterson             
                                                                               

delattr __slots__ inconsistancy                                     0 days
       http://bugs.python.org/issue7604    benjamin.peterson             
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.                0 days
       http://bugs.python.org/issue7608    alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                      1 days
       http://bugs.python.org/issue7609    benjamin.peterson             
       patch                                                                   

Fix "pass and object" typo in Library Reference / Built-in Types    0 days
       http://bugs.python.org/issue7612    ezio.melotti                  
       patch                                                                   

[cppcheck] found a regression : Invalid number of character (().    0 days
       http://bugs.python.org/issue7613    ezio.melotti                  
       patch                                                                   

Python 2.6.4 segfaults                                              0 days
       http://bugs.python.org/issue7614    mark.dickinson                
                                                                               

Fwd: Debian Bug#42318: urllib.py has problems with malformed pro 3436 days
       http://bugs.python.org/issue210849  shinnosuke                    
                                                                               

urllib / urllib2 should cache 301 redirections                   2425 days
       http://bugs.python.org/issue735515  pitrou                        
                                                                               

fast modular exponentiation                                      2084 days
       http://bugs.python.org/issue936813  mark.dickinson                
       patch                                                                   

bdist_deb - Debian packager                                       316 days
       http://bugs.python.org/issue1054967 astraw                        
       patch                                                                   

Carbon.Scrap.PutScrapFlavor                                       987 days
       http://bugs.python.org/issue1700507 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 11 Add Option to Bind to a Local IP Address in httplib.py           462 days
open    http://bugs.python.org/issue3972   

  8 fast modular exponentiation                                     2084 days
closed  http://bugs.python.org/issue936813 

  6 [patch] difflib should separate filename from timestamp with ta    5 days
open    http://bugs.python.org/issue7585   

  6 [patch] diff.py to use iso timestamp                               5 days
open    http://bugs.python.org/issue7582   

  6 Implement fastsearch algorithm for rfind/rindex                   24 days
open    http://bugs.python.org/issue7462   

  5 test_xmlrpc fails with non-ascii path                              2 days
open    http://bugs.python.org/issue7606   

  5 test_logging sometimes fails                                       1 days
closed  http://bugs.python.org/issue7596   

  5 Patch to add docstrings to msvcrt                                  6 days
open    http://bugs.python.org/issue7579   

  5 Distutils-based installer does not detect 64bit versions of Pyt  126 days
open    http://bugs.python.org/issue6792   

  5 _sha256 et al. encode to UTF-8 by default                         17 days
open    http://bugs.python.org/issue3745   




From stefan_ml at behnel.de  Sun Jan  3 09:11:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Jan 2010 09:11:16 +0100
Subject: [Python-Dev] Providing support files to assist 3.x extension
	authors
In-Reply-To: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
References: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
Message-ID: <hhpjf3$lk1$1@ger.gmane.org>

Case Vanhorsen, 20.12.2009 01:38:
> When I ported gmpy (Python to GMP multiple precision library) to
> Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I
> found the code to slightly faster and cleaner than using PyLong_AsLong
> and checking for overflow.

You might want to look at the code Cython generates for integer type 
conversions. We use specialised coercion code that gets generated 
on-the-fly to convert Python long/int from and to all sorts of C integer 
types with compile time (portability) and runtime size/value checks. 
Depending on your needs, this may or may not be faster than the above C-API 
function.

Stefan



From david.lyon at preisshare.net  Mon Jan  4 00:42:15 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 10:42:15 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
Message-ID: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>


> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>
> This new operator removes the ambiguity the original proposal had,
> without making it more
> complex for common use cases. So if you dislike it, you will need to
> propose something
> else that also fixes the ambiguity we had.

Ok.

> Environment markers
>..
>Here are some example of fields using such markers:
>
>Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'

  Requires-Dist: [Windows] pywin32 1.0+

That's simpler, shorter, and less ambiguous. Easier to
parse for package managers.

David





From python at mrabarnett.plus.com  Mon Jan  4 01:14:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 04 Jan 2010 00:14:43 +0000
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4132F3.7020905@mrabarnett.plus.com>

David Lyon wrote:
>> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>>
>> This new operator removes the ambiguity the original proposal had,
>> without making it more
>> complex for common use cases. So if you dislike it, you will need to
>> propose something
>> else that also fixes the ambiguity we had.
> 
> Ok.
> 
>> Environment markers
>> ..
>> Here are some example of fields using such markers:
>>
>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.
> 
'win32' is more specific than 'Windows' and, to me, '1.0+' means
'>=1.0', not '>1.0'.


From martin at v.loewis.de  Mon Jan  4 01:21:53 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 04 Jan 2010 01:21:53 +0100
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4134A1.5090203@v.loewis.de>

>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.

Don't you want the PEP to complete? Why this bike-shedding?

I can agree it's shorter. I can't agree that it's simpler,
or less ambiguous.

Regards,
Martin


From ssteinerx at gmail.com  Mon Jan  4 01:29:14 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 3 Jan 2010 19:29:14 -0500
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
	Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <DF433474-9F8E-40BA-B0B1-D3021CAC6317@gmail.com>


On Jan 3, 2010, at 7:21 PM, Martin v. L?wis wrote:

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>> 
>>  Requires-Dist: [Windows] pywin32 1.0+
>> 
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
> 
> Don't you want the PEP to complete? Why this bike-shedding?

Really....

Enough, already!

S



From david.lyon at preisshare.net  Mon Jan  4 01:47:56 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 11:47:56 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <1349.218.214.45.58.1262566076.squirrel@syd-srv02.ezyreg.com>


Hi Martin, Happy New Year,

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>>
>>   Requires-Dist: [Windows] pywin32 1.0+
>>
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
>
> Don't you want the PEP to complete? Why this bike-shedding?

Well, I'm just helping out by pointing out some simpler ways
as Tarek asked me. I was only answering his question. I was out
celebrating so it took longer to reply than normal.

Bike-Shedding ? Me ? which bikeshed? wanting simple?

Anyway, I'm just reading the PEPs and commenting. If there
are some alterations that can be done, lets discuss them.

> I can agree it's shorter.
> ..

Cool.

What I'd really like is a 'Code-Repository:' keyword
so that we can install programs/libraries directly into
a system.

I feel that this would really simplify the packaging
landscape, making it much easier for the scientific
community and others to do python software installs.

This would allow us to perphaps have something even
*more modern* than CPAN.

So yes, getting PEP-345 right is important to me.

Have a nice day.

David





From abpillai at gmail.com  Mon Jan  4 10:08:05 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Mon, 4 Jan 2010 14:38:05 +0530
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <8548c5f31001040108v635a78ech33521371c6c50e82@mail.gmail.com>

On Sun, Dec 27, 2009 at 11:21 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Tue, Dec 15, 2009 at 12:37 AM, Steven Bethard
> <steven.bethard at gmail.com> wrote:
> >
> > If you're only concerned about 2.X, then yes, optparse will *never* be
> > removed from 2.X. There will be a deprecation note in the 2.X
> > documentation but deprecation warnings will only be issued when the -3
> > flag is specified. Please see the "Deprecation of optparse" section of
> > the PEP:
> >
> > http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
>
 Ref http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse .

 Considering that optparse will be deprecated like 5 years from now.
 I think this point is moot. The deprecation strategy IMHO is
 perhaps way too conservative. Maybe it should be deprecated
 faster ;)




-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100104/e0cabd8c/attachment-0003.html>

From fetchinson at googlemail.com  Mon Jan  4 10:32:10 2010
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 4 Jan 2010 10:32:10 +0100
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <fbe2e2101001040132o22302d2ct72b705ec046bcc46@mail.gmail.com>

>> If you're only concerned about 2.X, then yes, optparse will *never* be
>> removed from 2.X. There will be a deprecation note in the 2.X
>> documentation but deprecation warnings will only be issued when the -3
>> flag is specified. Please see the "Deprecation of optparse" section of
>> the PEP:
>>
>> http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
> Compare:
> http://docs.python.org/library/optparse.html
> http://argparse.googlecode.com/svn/tags/r101/doc/index.html
>
> argparse should be recommended as advanced and more featured variant
> of optparse - that goes without saying, but forcing people to switch
> and annoying them with deprecation messages brings only headache.

As has been noted already nobody is forcing people to switch. Optparse
will be available as a separate package and everybody will be free to
install it and will not have any deprecation messages anywhere.

> Just
> like optparse is better getopt, the latter could also be useful for
> people coming from other languages and porting their libraries to
> Python.
>
> I would better concentrate on real code examples how argparse solves
> problems and would really want to see
> http://www.python.org/dev/peps/pep-0389/#out-of-scope-various-feature-requests
> implemented until argparse enters phase where backward incompatible
> changes in API are impossible.
>
> I would prefer to see PEP 389 as a document describing proposed
> solutions to argument parsing problems rather than petition to replace
> one library with another. So, it should display common argument
> parsing scenarios (use cases) with examples that are also useful
> recipes for documentation. I guess more than 90% people here doesn't
> have time to read argparse methods descriptions to see what they could
> be useful for.



-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From olemis at gmail.com  Mon Jan  4 15:24:05 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 4 Jan 2010 09:24:05 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
Message-ID: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>

On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
> Thanks for the quick response.
>
> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> When I made that change, I didn't know that the __unittest "hack" was
>> being used elsewhere outside of unittest, so I felt fine replacing it
>> with another. While I still consider it an implementation detail, I
>> would be ok with exposing an "official" API for this. Perhaps
>> __unittest_ignore_traceback?
>
> Well, bazaar has had the trick for a couple of years, and googling
> around now turns up some other projects using it or thinking about it:
> <http://github.com/gfxmonk/mocktest/commit/b5e94f7ee06ab627cea2c9cf90d0aeb63ee5a698>
> <http://bitbucket.org/uche/amara/changeset/eeaf69f48271/>
> <http://twistedmatrix.com/trac/ticket/4127>
>

Add `dutest` and probably `nose` [2]_ and ...

> Reinstating the old implementation (with the same name) would mean
> that existing code would keep working with Python 2.7

IOW ... if it is removed all the libraries will have to change and
check for the Py version and ... (probably not a big deal depending on
the details of the ?solution?)

> but maybe a
> discussion could start about a new, less hacky, way of doing the same
>

I am strongly -1 for modifying the classes in ?traditional? unittest
module [2]_ (except that I am strongly +1 for the package structure
WITHOUT TOUCHING anything else ...) , and the more I think about it I
am more convinced ... but anyway, this not a big deal (and in the end
what I think is not that relevant either ... :o). So ...

pass

> May not be worthwhile making life more complicated though,
> there aren't *that* many unittest-extending projects.
>

But every library extending `unittest` will do it or otherwise
not-so-useful error messages will be displayed
;o)

PS: Happy New Year ! BTW

.. [1] [feature] nosexml should omit trace frames where `__unittest...
        (http://code.google.com/p/python-nosexml/issues/detail?id=4#c0)

.. [2] Assessment of unittest 2.7 API : new features and opinions
        (http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free new ripit 1.3.3 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/KFwyUTKH0vI/


From juanfhj at gmail.com  Tue Jan  5 17:10:16 2010
From: juanfhj at gmail.com (Juan Fernando Herrera J.)
Date: Tue, 5 Jan 2010 11:10:16 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards compatibility
Message-ID: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>

How about a new python 3 release with (possibly partial) backwards
compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
software hasn't been ported to it. I'm eager to use 3, but paradoxically,
the 3 release makes me rather stuck with 2.6. Excuse me if this has been
suggested in the past.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/7839cf7b/attachment-0005.htm>

From fuzzyman at voidspace.org.uk  Tue Jan  5 17:50:15 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 05 Jan 2010 16:50:15 +0000
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <4B436DC7.8040605@voidspace.org.uk>

On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
> How about a new python 3 release with (possibly partial) backwards 
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way 
> major software hasn't been ported to it. I'm eager to use 3, but 
> paradoxically, the 3 release makes me rather stuck with 2.6. Excuse me 
> if this has been suggested in the past.
>    

I don't know about other developers, but I certainly expected Python 3 
adoption to take a few years due to libraries only gradually making the 
jump. I also expected there to be nervousness and pain around the 
migration as well.

I think in fact that libraries *are* migrating and there are lots of 
encouraging signs. As a community we need to do all we can to promote 
Python 3 adoption, but I don't think there is much reason to be worried 
(or complacent for that matter).

All the best,

Michael Foord

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/d7b6baac/attachment-0005.htm>

From brian.curtin at gmail.com  Tue Jan  5 18:21:31 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jan 2010 11:21:31 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>

On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:

> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
The proper route to take, in my opinion, is to see what 2.x libraries you
are using that are not 3.x compatible, run 2to3 on them, then run their test
suite, and see where you get. Submit a patch or two to the library and see
what happens -- it at least gets the wheels in motion.

I'm sure everyone out there would like to dive into supporting 3.x, but it's
tough to prioritize when you are up to your eyeballs with 2.x bugs in your
tracker. Look at Twisted (
http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3)
-- over 1000 issues, ~5 developers -- 3.x support won't be here tomorrow,
but it's on the horizon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/67a4e6e2/attachment-0005.htm>

From guido at python.org  Tue Jan  5 18:23:08 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jan 2010 09:23:08 -0800
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B436DC7.8040605@voidspace.org.uk>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<4B436DC7.8040605@voidspace.org.uk>
Message-ID: <ca471dc21001050923i2ca37f6ej543faf0981c43b13@mail.gmail.com>

On Tue, Jan 5, 2010 at 8:50 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
>
> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
> I don't know about other developers, but I certainly expected Python 3
> adoption to take a few years due to libraries only gradually making the
> jump. I also expected there to be nervousness and pain around the migration
> as well.
>
> I think in fact that libraries *are* migrating and there are lots of
> encouraging signs. As a community we need to do all we can to promote Python
> 3 adoption, but I don't think there is much reason to be worried (or
> complacent for that matter).
>

Michael is right, but doesn't answer the OP's implied question "why can't
3.x be made backwards compatible with 2.6?"

Unfortunately the answer isn't easy. I wish it was "because we didn't have
time to do it" (since then the solution would be simply to call for more
volunteers to help out) -- but unfortunately, it comes closer to "we have no
idea how to do it."

Py3k was designed to be backwards incompatible, because that gives the most
long-term benefit of the language improvements. (Perhaps a better way of
saying this is that in the design of language improvements,
feature-by-feature backwards compatibility was intentionally not a
requirement, even though keeping most of the language mostly unchanged *was*
a requirement.)

In principle it would be possible to be more backwards compatible at the
syntactic level, but that's not where the pain of porting code lies -- the
major hurdles are typically he new way of thinking about Unicode (bytes vs.
text strings, instead of 8-bit-strings vs. Unicode strings), and the changed
semantics of dictionary keys and related APIs. Since these issues typically
exist across modules (passing strings and dicts between modules is common),
recognizing individual modules as "2.x" vs. "3.x" isn't possible.

Note, by the way, that you won't be "stuck" on 2.6 forever -- 2.7 alpha 1
was already released.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/eaba2f9b/attachment-0005.htm>

From regebro at gmail.com  Tue Jan  5 18:52:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 18:52:20 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <319e029f1001050952o71cb29afh66445550beeb99c2@mail.gmail.com>

On Tue, Jan 5, 2010 at 17:10, Juan Fernando Herrera J.
<juanfhj at gmail.com> wrote:
> I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.

Yes. Python 3 is not what you want to use today if you write
applications. If you write libraries 2010 is the year to port, IMO.
With some luck, 2011 will be the year to start porting applications
properly. We'll see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ianb at colorstudy.com  Tue Jan  5 20:00:49 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 13:00:49 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
Message-ID: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>

On Tue, Jan 5, 2010 at 11:21 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:
>
>> How about a new python 3 release with (possibly partial) backwards
>> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
>> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
>> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
>> suggested in the past.
>>
>>
> The proper route to take, in my opinion, is to see what 2.x libraries you
> are using that are not 3.x compatible, run 2to3 on them, then run their test
> suite, and see where you get. Submit a patch or two to the library and see
> what happens -- it at least gets the wheels in motion.
>

It's not even that easy -- libraries can't apply patches for Python 3
compatibility as they usually break Python 2 compatibility.  Potentially
libraries could apply patches that make a codebase 2to3 ready, but from what
I've seen that's more black magic than straight forward updating, as such
patches have to trick 2to3 producing the output that is desired.

The only workable workflow I've seen people propose for maintaining a single
codebase with compatibility across both 2 and 3 is to use such tricks, with
aliases to suppress some 2to3 updates when they are inappropriate, so that
you can run 2to3 on install and have a single canonical Python 2 source.
 Python 2.7 won't help much (even though it is trying) as the introduction
of non-ambiguous constructions like b"" aren't compatible with previous
versions of Python and so can't be used in many libraries (support at least
back to Python 2.5 is the norm for most libraries, I think).

Also, running 2to3 on installation is kind of annoying, as you get source
that isn't itself the canonical source, so to fix bugs you have to look at
the installed source and trace it back to the bug in the original source.

I suspect a reasonable workflow might be possible with hg and maybe patch
queues, but I don't feel familiar enough with those tools to map that out.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/e43ccd78/attachment-0005.htm>

From glyph at twistedmatrix.com  Tue Jan  5 21:24:45 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 5 Jan 2010 15:24:45 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.

From regebro at gmail.com  Tue Jan  5 21:57:35 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 21:57:35 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <319e029f1001051257k3827c52ahcd115b15d9a8ece7@mail.gmail.com>

On Tue, Jan 5, 2010 at 21:24, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It seems like this is a problem to be addressed, then. ?Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

Some of it is about changing the code so 2to3 doesn't have to do ugly
things. For example, always use iteritems(), so that 2to3 knows that
items() can be used without converting it to a list, etc. Then we do
have the problems with unicode vs string vs bytes, where I can't think
of a clever solution except your no-op shims, which admittedly is
fugly . For me that tends to turn into testing everything until the
tests run on all platforms, which I guess is close to "black magic".
Don't know what to do about that.

python-incompatibility is about documenting all differences, and also
how to make code that runs under both 2.6 and 3.0 without 2to3. But I
guess it should be extended into how to spell something that is clean
in both 2.6 and 3.x.

>> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.

Ian: If you have examples of 2to3 updated that are not appropriate
(except the x.items() -> list(x.items()) they would be appreciated.

> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

In my experience it turned out to be less of a problem than I thought.
That is to some extent because the modules I've ported has had good
test coverage, and I have fixed 99.9% of the bugs by making the tests
pass. Developing with Distributes 2to3 support has then been quite
smooth and in most cases the separation between the 2.x source and the
3.x source hasn't been a problem.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan  5 22:07:23 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 22:07:23 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <4B43AA0B.5030308@v.loewis.de>

> It's not even that easy -- libraries can't apply patches for Python 3
> compatibility as they usually break Python 2 compatibility.  Potentially
> libraries could apply patches that make a codebase 2to3 ready, but from
> what I've seen that's more black magic than straight forward updating,
> as such patches have to trick 2to3 producing the output that is desired.

I wouldn't qualify it in that way. It may be necessary occasionally to
trick 2to3, but that's really a bug in 2to3 which you should report, so
that trickery is then a work-around for a bug - something that you may
have to do with other API, as well.

The "black magic" is really more in the parts that 2to3 doesn't touch
at all (because they are inherently not syntactic); these are the
problem areas Guido refers to. The "black magic" then is to make the
same code work unmodified for both 2.x and 3.x.

> The only workable workflow I've seen people propose for maintaining a
> single codebase with compatibility across both 2 and 3 is to use such
> tricks, with aliases to suppress some 2to3 updates when they are
> inappropriate

I think you misunderstand. All this is necessary, but *not* to suppress
2to3 updates. More typically, it is necessary because 2to3 leaves the
code unmodified either way.

> Also, running 2to3 on installation is kind of annoying, as you get
> source that isn't itself the canonical source, so to fix bugs you have
> to look at the installed source and trace it back to the bug in the
> original source.

That's an issue indeed, but one that I would hope that can be fixed by
improved traceback printing. It would be good if such traceback printing
could make it into 2.7.

Regards,
Martin


From martin at v.loewis.de  Tue Jan  5 22:13:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Jan 2010 22:13:07 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <4B43AB63.3000607@v.loewis.de>

> No-op constructions like 'bytes("")' could help for older versions of
> Python, though.  A very, very small runtime shim could provide
> support for these, if 2to3 could be told about it somehow.

You actually don't *need* 2to3 support for that - bytes("") can work
in either version:

2.x:
def bytes(s):
  return s

3.x:
def bytes(s)
  return s.encode("ascii")

On top of that, you *could* as for bytes("") to be converted to b"" in
3.x - and it is indeed possible to tell 2to3 about that, and you don't
even need to modify 2to3's source to do so: it can be part of your
package.

>> Also, running 2to3 on installation is kind of annoying, as you get
>> source that isn't itself the canonical source, so to fix bugs you
>> have to look at the installed source and trace it back to the bug
>> in the original source.
> 
> Given the way tracebacks are built, i.e. from filenames stored in
> .pycs rather than based on where the code was actually loaded in the
> filesystem, couldn't 2to3 could do .pyc rewriting to point at the
> original source?  Sort of like our own version of the #line
> directive? :)

I think it could, but it would be fairly flaky as the pycs
can get lost, and lose that information after regeneration. Also,
it may be confusing in other scenarios.

I'd rather have it generate separate mapper files, and change the
traceback printing to consider these (as an option).

> Seriously though, I find it hard to believe that this is a big
> problem.  The 3.x source looks pretty similar to the 2.x source, and
> it's good to look at both if you're dealing with a 3.x issue.

That's my experience as well - the only challenge is that you can't
cut-n-paste the source URL into an editor.

Regards,
Martin


From ianb at colorstudy.com  Tue Jan  5 22:39:21 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 15:39:21 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B43AA0B.5030308@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com> 
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com> 
	<4B43AA0B.5030308@v.loewis.de>
Message-ID: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>

On Tue, Jan 5, 2010 at 3:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > It's not even that easy -- libraries can't apply patches for Python 3
> > compatibility as they usually break Python 2 compatibility. ?Potentially
> > libraries could apply patches that make a codebase 2to3 ready, but from
> > what I've seen that's more black magic than straight forward updating,
> > as such patches have to trick 2to3 producing the output that is desired.
>
> I wouldn't qualify it in that way. It may be necessary occasionally to
> trick 2to3, but that's really a bug in 2to3 which you should report, so
> that trickery is then a work-around for a bug - something that you may
> have to do with other API, as well.
>
> The "black magic" is really more in the parts that 2to3 doesn't touch
> at all (because they are inherently not syntactic); these are the
> problem areas Guido refers to. The "black magic" then is to make the
> same code work unmodified for both 2.x and 3.x.

Just to clarify, the black magic I'm referring to is things like:

try:
?? ?unicode_ = unicode
except NameError:
?? ?unicode_ = str

and some other aliases like this that are unambiguous and which 2to3
won't touch (if you write them correctly). ?If the porting guide noted
all these tricks (of which several have been developed, and I'm only
vaguely aware of a few) that would be helpful. ?It's not a lot of
tricks, but the tricks are not obvious and 2to3 gets the translation
wrong pretty often without them. ?For instance, when I say str in
Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
str and unicode to str. ?That *nothing* translates to bytes by default
(AFAIK) means that people must either be living in a bytes-free world
(which sure, lots of code does) or they are using tricks not included
in 2to3 itself.


Also replying to Glyph:
> > Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.
>
> Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in
the filesystem, couldn't 2to3 could do .pyc rewriting to point at the
original source? ?Sort of like our own version of the #line directive?
:)
>
> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
then I don't currently develop any code that is "installed", I only
develop code that is directly from a source code checkout, and where
the checkout is put on the path.  I guess I could have something that
automatically builds the code on every edit, and that's not
infeasible.  It's just not fun.  So long as I have to support Python 2
(which is like forever) then adding Python 3 only makes development
that much more complicated and much less fun, with no concrete
benefits.  Which is a terribly crotchety of me.  Sorry.

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker


From martin at v.loewis.de  Tue Jan  5 23:23:53 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 23:23:53 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
Message-ID: <4B43BBF9.2000302@v.loewis.de>

> Just to clarify, the black magic I'm referring to is things like:
> 
> try:
>     unicode_ = unicode
> except NameError:
>     unicode_ = str
> 
> and some other aliases like this that are unambiguous and which 2to3
> won't touch (if you write them correctly).  If the porting guide noted
> all these tricks (of which several have been developed, and I'm only
> vaguely aware of a few) that would be helpful.  It's not a lot of
> tricks, but the tricks are not obvious and 2to3 gets the translation
> wrong pretty often without them.  For instance, when I say str in
> Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
> str and unicode to str.

No, that's not what's happening. Instead, str is not translated at all
(i.e. it's *not* translated to str - it just isn't touched).

Translating unicode to str is always correct, AFAICT.

I'm not quite sure what the magic above is supposed to achieve: in 2.x,
unicode_ becomes an alias to unicode, in 3.x, 2to3 translates unicode
to str, and then the block becomes

try:
  unicode_ = str
except NameError:
  unicode_ = str

so the NameError is actually never triggered.

Could it be that the magic is proposed for a mode where you *don't*
use 2to3?

> That *nothing* translates to bytes by default
> (AFAIK) means that people must either be living in a bytes-free world
> (which sure, lots of code does) or they are using tricks not included
> in 2to3 itself.

By your above definition of "translates", the function "bytes" is
translated to bytes (i.e. it isn't touched by 2to3).

> Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
> then I don't currently develop any code that is "installed", I only
> develop code that is directly from a source code checkout, and where
> the checkout is put on the path.  I guess I could have something that
> automatically builds the code on every edit, and that's not
> infeasible.  It's just not fun.

Depends on where you draw your fun from. It is indeed fun to me, but
I can see why it may not be fun to you. So you could ask me to do it for
you - or you may try to use what I have already done for you, so you
don't have to do it.

> So long as I have to support Python 2
> (which is like forever) then adding Python 3 only makes development
> that much more complicated and much less fun, with no concrete
> benefits.

I doubt it will be *much* more complicated - only a little.
As for concrete benefits: there may be no benefits at this point,
but in the long run, starting now will have saved you a lot of
pressure in the long run, and stop users from switching away from
your packages because of lack of Python 3 support.

Regards,
Martin


From ziade.tarek at gmail.com  Wed Jan  6 01:08:34 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 01:08:34 +0100
Subject: [Python-Dev] PEP 386 and PEP 345
Message-ID: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>

Hi,

I think we've reached a consensus on those two PEPs.

Although, there's one last point that was forgotten in the discussions
: I've introduced "rc" in the pre-releases markers, so PEP 386 is
compatible with Python's own version scheme.  "rc" comes right after
"c" in the sorting. It's slightly redundant with the "c" marker but I
don't think this really matters as long as consumers know how to order
them (a < b < c < rc). I have also stated that "c" is the preferred
marker for third party projects, from PEP 386 point of view.

Is there anything else I can do to make those two PEPs accepted ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From david.lyon at preisshare.net  Wed Jan  6 01:26:34 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:26:34 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <4B43BBF9.2000302@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
	<4B43BBF9.2000302@v.loewis.de>
Message-ID: <1322.218.214.45.58.1262737594.squirrel@syd-srv02.ezyreg.com>

Hi Martin,

> ... but in the long run, starting now will have saved you a lot of
> pressure in the long run, and stop users from switching away from
> your packages because of lack of Python 3 support.

In a production situation it works the other way around. If there's
an application that requires twisted (or whatever package) then most
people would use the appropriate interpreter to match the library.

Since you guys all did your jobs so well :-) doing so is painless.

Because there is so much "comfort" with the existing situation it
makes it an effort for people to move to a different lounge chair.
Namely python 3.

I'd suggest that moving the package set (pypi) to python 3 could
be kicked along with the help of some automated tools. I don't
know what tools you guys have got.

But I am very sure that if code analysis was provided to package
developers on python 3 (so they don't have to run it themselves),
then it would be like an even bigger tv screen in a bigger lounge
room and it would assist in drawing them over.

David







From david.lyon at preisshare.net  Wed Jan  6 01:36:24 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:36:24 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <1337.218.214.45.58.1262738184.squirrel@syd-srv02.ezyreg.com>


Hi Ian,

> The only workable workflow I've seen people propose for maintaining a
> single
> codebase with compatibility across both 2 and 3 is to use such tricks,
> with
> aliases to suppress some 2to3 updates when they are inappropriate, so that
> you can run 2to3 on install and have a single canonical Python 2 source.
>  Python 2.7 won't help much (even though it is trying) as the introduction
> of non-ambiguous constructions like b"" aren't compatible with previous
> versions of Python and so can't be used in many libraries (support at
> least
> back to Python 2.5 is the norm for most libraries, I think).
>
> Also, running 2to3 on installation is kind of annoying, as you get source
> that isn't itself the canonical source, so to fix bugs you have to look at
> the installed source and trace it back to the bug in the original source.
>
> I suspect a reasonable workflow might be possible with hg and maybe patch
> queues, but I don't feel familiar enough with those tools to map that out.

That's why I am saying that we need a Code-Repository: section in PEP-345
Metadata.

Let package developers keep two branches in SCM. One for 2.x and one for 3.x

Let them backport features from their 3.x development series to their 2.x
code base. In exactly the same way that it is done in so many other
developments today.

Keeping multiple branches of code is a very common thing these days. And
there are tools in the outside world (bzr,svn,mercurial) that allow us to
do it very easily.

Let's have that support in python; it will make life easier.

David










From david.lyon at preisshare.net  Wed Jan  6 03:01:22 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 13:01:22 +1100 (EST)
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <1617.218.214.45.58.1262743282.squirrel@syd-srv02.ezyreg.com>

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?

Tarek,

Given that I helped out so much last year with the PEP in discussing
different options, even if they weren't accepted, I really feel that it is
unfair if my name isn't mentioned. It was a huge time sacrifice on my
part.

For example, even if I only managed to explain the version numbering and
clarify how that worked. It did take me some time to do that.

What I did do however, was spend a lot of time with the multiplatform
"Markers". I still think that two short weeks more of "discussion" could
resolve some issues. That discussion went for 4 months on distutils-ml.

Look, major issues aside, can you make the following concessions on
PEP-345 which I only feel will help it, namely:

 1) Source-Repository: specify a code repository to install from

 2) Streamline Requires-Python: by implementing "Markers" as
    noted by the PEP. A marker being something like
    "Requires-Python(windows): lxml". Otherwise remove the
    word marker from the PEP and just replace with "metacode".
    Markers are what were discussed on distutils-ml. Metacode
    is what is described in the PEP.

 3) Remove the inconsistency and platform ambiguities. Especially
    for windows users. For example, "win32" is extremely confusing
    for windows users right now. As more and more systems now are
    64 bit. Use the platform module, instead of the sys module
    for constants. I'll post to distutils-ml on this.

I am certainly not trying to hold this PEP up, and I apologise on
my part for my late attention. I will post to distutils-ml on these
and i promise to keep my comments unheated and unwitty.

Having said that, PEP-345 is *super-important*. A week or two or three
more discussion and the issues can be resolved.

We all just want to focus on being productive. It would be a great
accomplishment for you to get PEP-345 approved and likewise for
me getting mentioned even in a minor role as helping out on a PEP.

So I'm hoping that you can make a few last minute concessions
meaning that we can all happily go on our way in 2010.

Regards

David













From brett at python.org  Wed Jan  6 06:20:30 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Jan 2010 21:20:30 -0800
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <bbaeab101001052120qe888df8o7eb03a61e6c030c7@mail.gmail.com>

On Tue, Jan 5, 2010 at 16:08, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?
>

As you said, consensus has been reached, so just Guido's BDFL stamp of
approval is all I can think of.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/a76cb75b/attachment-0005.htm>

From chris at simplistix.co.uk  Wed Jan  6 12:19:45 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:19:45 +0000
Subject: [Python-Dev] bug triage
Message-ID: <4B4471D1.9020707@simplistix.co.uk>

Hi All,

Is there a high volume of incoming bugs to the Python tracker?
If so, I'd like to help with triaging. I think I have all the necessary 
access, what I'm missing is the knowledge of how to set myself up to get 
notifications of new bugs...

How do I do that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From fuzzyman at voidspace.org.uk  Wed Jan  6 12:24:37 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 06 Jan 2010 11:24:37 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <4B4472F5.8000709@voidspace.org.uk>

On 06/01/2010 11:19, Chris Withers wrote:
> Hi All,
>
> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the 
> necessary access, what I'm missing is the knowledge of how to set 
> myself up to get notifications of new bugs...
>
> How do I do that?
>

Bug triaging is one of Python's "big needs" and anything you do to help 
on this score would be much appreciated. Particularly reviewing new and 
outstanding issues.

I assumed there would be RSS feeds for bug tracker activity but can't 
easily find these on the tracker. There is a bot that posts activity to 
#python-dev, so there must be some way of getting this information.

All the best,

Michael



> cheers,
>
> Chris
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From solipsis at pitrou.net  Wed Jan  6 12:25:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 11:25:16 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <loom.20100106T122258-896@post.gmane.org>


Hi Chris,

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary 
> access, what I'm missing is the knowledge of how to set myself up to get 
> notifications of new bugs...

Do you really want to get such notifications? There may be a lot of them.
If you want however, you can join #python-dev on IRC (irc.freenode.net) where
there's a bot which posts updates of all bugs on the tracker. There's usually
not a lot of discussion going on so you probably won't feel flooded.

In addition to bug triage, what is needed is reviewing of existing patches, as
well as writing patches for issues which haven't been addressed yet :-)

Regards

Antoine.




From chris at simplistix.co.uk  Wed Jan  6 12:30:40 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:30:40 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B447460.7080100@simplistix.co.uk>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't 
> easily find these on the tracker. There is a bot that posts activity to 
> #python-dev, so there must be some way of getting this information.

Yeah, email-out is what I'm really after... I have it for my own Roundup 
instance so it can't be that hard to do ;-)

Roch?, you guys host the bug tracker, right? Is there email-out set up 
for it?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ncoghlan at gmail.com  Wed Jan  6 12:37:23 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 21:37:23 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B4475F3.5040406@gmail.com>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't
> easily find these on the tracker. There is a bot that posts activity to
> #python-dev, so there must be some way of getting this information.

I'm pretty sure the bugs list is still the primary spooled notification
mechanism:
http://mail.python.org/mailman/listinfo/python-bugs-list

There are also the weekly tracker activity summaries that are posted
here to python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From chris at simplistix.co.uk  Wed Jan  6 12:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:41:28 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <4B4476E8.8050709@simplistix.co.uk>

Nick Coghlan wrote:
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

That's what I was after, thanks!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From facundobatista at gmail.com  Wed Jan  6 13:03:08 2010
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 6 Jan 2010 09:03:08 -0300
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <e04bdf311001060403y3b65c647jf82fdc26266a0efe@mail.gmail.com>

On Wed, Jan 6, 2010 at 8:19 AM, Chris Withers <chris at simplistix.co.uk> wrote:

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary
> access, what I'm missing is the knowledge of how to set myself up to get
> notifications of new bugs...

Not notifications, but maybe a way to have a higher look of them for
easy selection:

  http://www.taniquetil.com.ar/cgi-bin/pytickets.py

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ziade.tarek at gmail.com  Wed Jan  6 13:29:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:29:58 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>

On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 06/01/2010 11:19, Chris Withers wrote:
>>
>> Hi All,
>>
>> Is there a high volume of incoming bugs to the Python tracker?
>> If so, I'd like to help with triaging. I think I have all the necessary
>> access, what I'm missing is the knowledge of how to set myself up to get
>> notifications of new bugs...
>>
>> How do I do that?
>>
>
> Bug triaging is one of Python's "big needs" and anything you do to help on
> this score would be much appreciated. Particularly reviewing new and
> outstanding issues.

Another useful triage I think, is to review the oldest bugs (some of
them are > 5 years)
and remove the ones that are not relevant anymore, or duplicate with
newer entries.

Tarek

-- 
Tarek Ziad? | http://ziade.org


From chris at simplistix.co.uk  Wed Jan  6 13:31:08 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 12:31:08 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B44828C.4070201@simplistix.co.uk>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this 
with someone as a paired task for those 2 days...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Wed Jan  6 13:37:55 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:37:55 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B44828C.4070201@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B44828C.4070201@simplistix.co.uk>
Message-ID: <94bdd2611001060437s2d0242aembc669c3263773fea@mail.gmail.com>

On Wed, Jan 6, 2010 at 1:31 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Another useful triage I think, is to review the oldest bugs (some of
>> them are > 5 years)
>> and remove the ones that are not relevant anymore, or duplicate with
>> newer entries.
>
> I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this with
> someone as a paired task for those 2 days...

I'll be doing Distutils stuff but I can probably help around a bit in that task:
Distutils have quite a few old issues so I can tackle those


From roche at upfrontsystems.co.za  Wed Jan  6 13:32:39 2010
From: roche at upfrontsystems.co.za (=?ISO-8859-1?Q?Roch=E9?= Compaan)
Date: Wed, 06 Jan 2010 14:32:39 +0200
Subject: [Python-Dev] bug triage
In-Reply-To: <4B447460.7080100@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B447460.7080100@simplistix.co.uk>
Message-ID: <1262781159.2836.218.camel@didi>

On Wed, 2010-01-06 at 11:30 +0000, Chris Withers wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't 
> > easily find these on the tracker. There is a bot that posts activity to 
> > #python-dev, so there must be some way of getting this information.
> 
> Yeah, email-out is what I'm really after... I have it for my own Roundup 
> instance so it can't be that hard to do ;-)
> 
> Roch?, you guys host the bug tracker, right? Is there email-out set up 
> for it?

We do, but we don't administer it. There are a few administrators taking
care of it and you should be able to reach them by logging your request
here: http://psf.upfronthosting.co.za/roundup/meta/ or post it to the
Infrastructure mailing list: infrastructure at python.org


-- 
Roch? Compaan
Upfront Systems                   http://www.upfrontsystems.co.za



From ncoghlan at gmail.com  Wed Jan  6 13:57:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 22:57:24 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B4488B4.2070208@gmail.com>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I believe someone (Daniel Diniz, maybe?) did do a pass over those some
time in the  last 12 months, so most of the obviously irrelevant ones
that are that old should already be gone. Not to say it isn't worth
doing another pass, just saying not to get disheartened if there aren't
many that can be readily closed.

There are at least a few still kicking around just because they're
difficult to deal with (there's an ancient one to do with one of the
ways circular imports can fail that I occasionally go back and reread
before moving on to something more tractable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rdmurray at bitdance.com  Wed Jan  6 14:43:17 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 08:43:17 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4476E8.8050709@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
	<4B4476E8.8050709@simplistix.co.uk>
Message-ID: <20100106134317.3EF141FB5A6@kimball.webabinitio.net>

On Wed, 06 Jan 2010 11:41:28 +0000, Chris Withers <chris at simplistix.co.uk> wrote:
> Nick Coghlan wrote:
> > I'm pretty sure the bugs list is still the primary spooled notification
> > mechanism:
> > http://mail.python.org/mailman/listinfo/python-bugs-list
> 
> That's what I was after, thanks!

Just for completeness, there's also new-bugs-announce if you want
just *new* bug notification.  That's more for people who want to
watch for bugs they want to become nosy on, though; if you are
doing triage python-bugs-list is what you want.

Please also read http://www.python.org/dev/workflow/ if you haven't
already.  Thanks for being willing to chip in!

--David


From brian.curtin at gmail.com  Wed Jan  6 15:57:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 08:57:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4488B4.2070208@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
Message-ID: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I believe someone (Daniel Diniz, maybe?) did do a pass over those some
> time in the  last 12 months, so most of the obviously irrelevant ones
> that are that old should already be gone. Not to say it isn't worth
> doing another pass, just saying not to get disheartened if there aren't
> many that can be readily closed.
>
> There are at least a few still kicking around just because they're
> difficult to deal with (there's an ancient one to do with one of the
> ways circular imports can fail that I occasionally go back and reread
> before moving on to something more tractable).
>
> Cheers,
> Nick.
>

On the topic of bugs that can be readily closed (literally), I've recently
come across a number of issues which appear to be sitting in a patch or
review stage, but their patches have been committed and the issue remains
open. What is the best course of action there? I'd just go ahead and close
the issue myself but I don't have tracker privileges.

I'm willing to help out with another Daniel Diniz-esque triage sweep if that
would help.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/05d9ebd7/attachment-0005.htm>

From ssteinerx at gmail.com  Wed Jan  6 16:14:20 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 6 Jan 2010 10:14:20 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <7FCF8B19-3465-4392-80CC-BEAEBD926ECA@gmail.com>


On Jan 6, 2010, at 7:29 AM, Tarek Ziad? wrote:

> On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> On 06/01/2010 11:19, Chris Withers wrote:
>>> 
>>> Hi All,
>>> 
>>> Is there a high volume of incoming bugs to the Python tracker?
>>> If so, I'd like to help with triaging. I think I have all the necessary
>>> access, what I'm missing is the knowledge of how to set myself up to get
>>> notifications of new bugs...
>>> 
>>> How do I do that?
>>> 
>> 
>> Bug triaging is one of Python's "big needs" and anything you do to help on
>> this score would be much appreciated. Particularly reviewing new and
>> outstanding issues.
> 
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I was actually thinking about that the other day when I saw that the average age of bugs on the Python tracker was at some hideously large 3 digit number.  

The 'success' statistic would be to bring that down below, say, 100.

S



From skip at pobox.com  Wed Jan  6 17:38:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 6 Jan 2010 10:38:13 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <19268.48245.616046.874126@montanaro.dyndns.org>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> I'm pretty sure the bugs list is still the primary spooled
    Nick> notification mechanism:
    Nick> http://mail.python.org/mailman/listinfo/python-bugs-list

Actually, there is a new-bugs-announce list:

    http://mail.python.org/mailman/listinfo/new-bugs-announce

Skip


From solipsis at pitrou.net  Wed Jan  6 18:56:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 17:56:49 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <hi2it0$q48$1@ger.gmane.org>

Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> On the topic of bugs that can be readily closed (literally), I've
> recently come across a number of issues which appear to be sitting in a
> patch or review stage, but their patches have been committed and the
> issue remains open. What is the best course of action there?

Post a message on the issue asking for info.




From solipsis at pitrou.net  Wed Jan  6 19:09:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 18:09:39 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<hi2it0$q48$1@ger.gmane.org>
Message-ID: <loom.20100106T190650-337@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> > On the topic of bugs that can be readily closed (literally), I've
> > recently come across a number of issues which appear to be sitting in a
> > patch or review stage, but their patches have been committed and the
> > issue remains open. What is the best course of action there?
> 
> Post a message on the issue asking for info.

Ok, I realize my answer might have been a bit terse :-)
The patch might be waiting to be merged in all development branches, or it may
not totally resolve the issue, or perhaps documentation needs to be updated, or
perhaps it is pending a verdict from the buildbots, etc. You can't deduce that
the issue is completely fixed from the simple fact that something has been
committed.

Regards

Antoine Pitrou.






From brett at python.org  Wed Jan  6 20:03:32 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 11:03:32 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>  I believe someone (Daniel Diniz, maybe?) did do a pass over those some
>> time in the  last 12 months, so most of the obviously irrelevant ones
>> that are that old should already be gone. Not to say it isn't worth
>> doing another pass, just saying not to get disheartened if there aren't
>> many that can be readily closed.
>>
>> There are at least a few still kicking around just because they're
>> difficult to deal with (there's an ancient one to do with one of the
>> ways circular imports can fail that I occasionally go back and reread
>> before moving on to something more tractable).
>>
>> Cheers,
>> Nick.
>>
>
> On the topic of bugs that can be readily closed (literally), I've recently
> come across a number of issues which appear to be sitting in a patch or
> review stage, but their patches have been committed and the issue remains
> open. What is the best course of action there? I'd just go ahead and close
> the issue myself but I don't have tracker privileges.
>
>
If a core developer is willing to step forward and vouch for you to get
tracker privileges then I will give them to you. We are trying to give out
tracker privs w/ less time than required to get commit privileges. So as
long as you have helped out on a few issues in a positive and correct way
that should be enough to get one of the regulars who perform triage to
notice.

-Brett



I'm willing to help out with another Daniel Diniz-esque triage sweep if that
> would help.
>
> Brian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/f24c9595/attachment-0005.htm>

From nad at acm.org  Wed Jan  6 21:41:05 2010
From: nad at acm.org (Ned Deily)
Date: Wed, 06 Jan 2010 12:41:05 -0800
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <nad-19B167.12410506012010@news.gmane.org>

In article <4B4475F3.5040406 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't
> > easily find these on the tracker. There is a bot that posts activity to
> > #python-dev, so there must be some way of getting this information.
> 
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

Also, that mailing list (along with most python development related 
mailing lists) is mirrored at gmane.org which means it can also be 
obtained via a newsreader (NNTP) or various RSS feeds.

http://dir.gmane.org/gmane.comp.python.bugs

-- 
 Ned Deily,
 nad at acm.org



From nick.bastin at gmail.com  Wed Jan  6 22:14:54 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:14:54 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
Message-ID: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>

(This may occur on more platforms - I can test on more unix platforms
if the consensus is this is an actual problem and I'm not just a nut)

On freebsd5, if you do a simple ./configure --enable-shared in current
(2.7) trunk, your python shared library will build properly, but all
modules will fail to find the shared library and thus fail to build:

gcc -shared build/temp.freebsd-5.3-RELEASE-i386-2.7/u1/Python/Python-2.7a1/Modules/_struct.o
   -L/u1/tmp/python2.7a1/lib -L/usr/local/lib -lpython2.7 -o
build/lib.freebsd-5.3-RELEASE-i386-2.7/_struct.so
/usr/bin/ld: cannot find -lpython2.7
building '_ctypes_test' extension
...

This of course is because libpython2.7.so is in the current directory
and not (yet) installed in /usr/local/lib.  I've made a very simple
fix for this problem that works, but at least to me smells a bit
funny, which is to modify setup.py to add the following to
detect_modules():

        # If we did --enable-shared, we need to be able to find the library
        # we just built in order to build the modules.
        if platform == 'freebsd5':
            add_dir_to_list(self.compiler_obj.library_dirs, '.')


Which brings me to a few questions:

a) Does this seem like a real problem, or am I missing something obvious?

b) Does this fix seem like the sensible thing to do?  (it seems at
least that we ought to check that the user configured --enable-shared
and only set -L. in that case, if that's possible)

Setting --enable-shared when you actually have a libpython2.7.so in
/usr/local/lib (or whatever --prefix you've selected) is possibly even
more dangerous, because it may succeed in linking against a
differently-built library than what you intended.

--
Nick


From nick.bastin at gmail.com  Wed Jan  6 22:39:17 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:39:17 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <66d0a6e11001061339t38202b70mca1091e1926ecf4b@mail.gmail.com>

On Wed, Jan 6, 2010 at 16:14, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> This of course is because libpython2.7.so is in the current directory
> and not (yet) installed in /usr/local/lib.

One minor correction - as you could see from the compile line, the
actual --prefix in this case is /u1/tmp/python2.7a1, but the libraries
obviously aren't installed there yet either.  Perhaps a better fix
than setting -L. would be to put the shared library in
build/lib.freebsd-5.3-RELEASE-i386-2.7 and add that to the library
path for the linker (the build creates this directory, but installs
nothing in it).

--
Nick


From martin at v.loewis.de  Wed Jan  6 23:21:44 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Jan 2010 23:21:44 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <4B450CF8.8090009@v.loewis.de>

> b) Does this fix seem like the sensible thing to do?

No. Linking in setup.py should use the same options as if the module
was built as *shared* through Modules/Setup, which, IIUC, should use
BLDLIBRARY.

Regards,
Martin


From nick.bastin at gmail.com  Thu Jan  7 01:08:13 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 19:08:13 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B450CF8.8090009@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
Message-ID: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:21, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> b) Does this fix seem like the sensible thing to do?
>
> No. Linking in setup.py should use the same options as if the module
> was built as *shared* through Modules/Setup, which, IIUC, should use
> BLDLIBRARY.

Thanks for that pointer, that makes much more sense.  Indeed,
BLDLIBRARY on FreeBSD* is set to '-L. -lpython$(VERSION)' if you set
--enable-shared, but somehow that piece of information doesn't
propagate into the module build.  More investigation to be done...

--
Nick


From rdmurray at bitdance.com  Thu Jan  7 02:22:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 20:22:42 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
Message-ID: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>


On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On the topic of bugs that can be readily closed (literally), I've recently
> > come across a number of issues which appear to be sitting in a patch or
> > review stage, but their patches have been committed and the issue remains
> > open. What is the best course of action there? I'd just go ahead and close
> > the issue myself but I don't have tracker privileges.
> >
> >
> If a core developer is willing to step forward and vouch for you to get
> tracker privileges then I will give them to you. We are trying to give out
> tracker privs w/ less time than required to get commit privileges. So as
> long as you have helped out on a few issues in a positive and correct way
> that should be enough to get one of the regulars who perform triage to
> notice.
> 
> -Brett

I've done a quick scan of issues Brian is nosy on to refresh my
memory, and I'd say he's definitely been making positive contributions.
I'm willing to volunteer to keep an eye on his triage work for a while
if you grant him tracker privs.

Brian, I assume you'll be cognizant of Antoine's advice about making
sure a bug really should be closed before closing it :)  Hanging out in
#python-dev on freenode while working on issues can be helpful, as well,
since you can quickly ask whoever is there for second opinions on
particular bugs.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Thu Jan  7 02:28:26 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 17:28:26 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com> 
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com> 
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
Message-ID: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com> wrote:

>
> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On the topic of bugs that can be readily closed (literally), I've
> recently
> > > come across a number of issues which appear to be sitting in a patch or
> > > review stage, but their patches have been committed and the issue
> remains
> > > open. What is the best course of action there? I'd just go ahead and
> close
> > > the issue myself but I don't have tracker privileges.
> > >
> > >
> > If a core developer is willing to step forward and vouch for you to get
> > tracker privileges then I will give them to you. We are trying to give
> out
> > tracker privs w/ less time than required to get commit privileges. So as
> > long as you have helped out on a few issues in a positive and correct way
> > that should be enough to get one of the regulars who perform triage to
> > notice.
> >
> > -Brett
>
> I've done a quick scan of issues Brian is nosy on to refresh my
> memory, and I'd say he's definitely been making positive contributions.
> I'm willing to volunteer to keep an eye on his triage work for a while
> if you grant him tracker privs.
>
>
Done for the username brian.curtin (email doesn't match the one Brian
emailed from so do let me know, Brian if this is the right username).
Welcome aboard!

-Brett


> Brian, I assume you'll be cognizant of Antoine's advice about making
> sure a bug really should be closed before closing it :)  Hanging out in
> #python-dev on freenode while working on issues can be helpful, as well,
> since you can quickly ask whoever is there for second opinions on
> particular bugs.
>
> --
> R. David Murray                                      www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/726f5ca0/attachment-0005.htm>

From brian.curtin at gmail.com  Thu Jan  7 02:59:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 19:59:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
	<bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
Message-ID: <cf9f31f21001061759m4fee1cc7g2d9fe8bbfd3cb38e@mail.gmail.com>

On Wed, Jan 6, 2010 at 19:28, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com>wrote:
>
>>
>> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
>> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
>> wrote:
>> > > On the topic of bugs that can be readily closed (literally), I've
>> recently
>> > > come across a number of issues which appear to be sitting in a patch
>> or
>> > > review stage, but their patches have been committed and the issue
>> remains
>> > > open. What is the best course of action there? I'd just go ahead and
>> close
>> > > the issue myself but I don't have tracker privileges.
>> > >
>> > >
>> > If a core developer is willing to step forward and vouch for you to get
>> > tracker privileges then I will give them to you. We are trying to give
>> out
>> > tracker privs w/ less time than required to get commit privileges. So as
>> > long as you have helped out on a few issues in a positive and correct
>> way
>> > that should be enough to get one of the regulars who perform triage to
>> > notice.
>> >
>> > -Brett
>>
>> I've done a quick scan of issues Brian is nosy on to refresh my
>> memory, and I'd say he's definitely been making positive contributions.
>> I'm willing to volunteer to keep an eye on his triage work for a while
>> if you grant him tracker privs.
>>
>>
> Done for the username brian.curtin (email doesn't match the one Brian
> emailed from so do let me know, Brian if this is the right username).
> Welcome aboard!
>
>
Yep, that's the one. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/62a95fa0/attachment-0005.htm>

From python at mrabarnett.plus.com  Thu Jan  7 04:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 07 Jan 2010 03:07:56 +0000
Subject: [Python-Dev] GIL required for _all_ Python calls?
Message-ID: <4B45500C.8090905@mrabarnett.plus.com>

Hi,

I've been wondering whether it's possible to release the GIL in the
regex engine during matching.

I know that it needs to have the GIL during memory-management calls, but
does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
an easy way to find out? Or is it just a case of checking the source
files for mentions of the GIL? The header file for PyList_New, for
example, doesn't mention it!

Thanks


From john.arbash.meinel at gmail.com  Thu Jan  7 04:25:48 2010
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 06 Jan 2010 21:25:48 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B45543C.2090200@gmail.com>

MRAB wrote:
> Hi,
> 
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out? Or is it just a case of checking the source
> files for mentions of the GIL? The header file for PyList_New, for
> example, doesn't mention it!
> 
> Thanks

Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
get concurrent updating of the value, and then the final value is wrong.
(two threads do 5+1 getting 6, rather than 7, and when the decref, you
end up at 4 rather than back at 5).

AFAIK, the only things that don't require the GIL are macro functions,
like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
example, will be increfing and setting the exception state, so certainly
needs the GIL to be held.

John
=:->



From benjamin at python.org  Thu Jan  7 04:32:17 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jan 2010 21:32:17 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45543C.2090200@gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com> <4B45543C.2090200@gmail.com>
Message-ID: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>

2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
> get concurrent updating of the value, and then the final value is wrong.
> (two threads do 5+1 getting 6, rather than 7, and when the decref, you
> end up at 4 rather than back at 5).

Correct.

>
> AFAIK, the only things that don't require the GIL are macro functions,
> like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> example, will be increfing and setting the exception state, so certainly
> needs the GIL to be held.

As a general rule, I would say, no Py* macros are safe without the gil
either (the exception being Py_END_ALLOW_THREADS), since they mutate
Python objects which must be protected.



-- 
Regards,
Benjamin


From guido at python.org  Thu Jan  7 05:29:03 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Jan 2010 20:29:03 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B45543C.2090200@gmail.com> 
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
Message-ID: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>

On Wed, Jan 6, 2010 at 7:32 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> > AFAIK, the only things that don't require the GIL are macro functions,
> > like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> > example, will be increfing and setting the exception state, so certainly
> > needs the GIL to be held.
>
> As a general rule, I would say, no Py* macros are safe without the gil
> either (the exception being Py_END_ALLOW_THREADS), since they mutate
> Python objects which must be protected.

That's keeping it on the safe side, since there are some macros like
PyString_AS_STRING() that are also safe, *if* you are owning at least
one reference to the string object.

At the same time, "no Py* macros" is not quite strong enough, since if
you called PyString_AS_STRING() before releasing the GIL but you don't
own a reference to the string object, the string might be deallocated
behind your back by another thread.

A better rule would be "you may access the memory buffer in a PyString
or PyUnicode object with the GIL released as long as you own a
reference to the string object." Everything else is out of bounds (or
not worth the bother).

--
--Guido van Rossum (python.org/~guido)


From yoann.padioleau at facebook.com  Thu Jan  7 08:24:42 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Wed, 6 Jan 2010 23:24:42 -0800
Subject: [Python-Dev] relation between Python.asdl and Tools/compiler/ast.txt
Message-ID: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>

Hi,

I would like to use astgen.py to generate python classes corresponding to the 
AST of something I have defined in a .asdl file, along the line of what is
apparently done for the python AST itself. I thought astgen.py would
take as an argument a .asdl file, but apparently it instead process a file
called ast.txt. Where does this file come from ? Is it generated from
Python.asdl ?



From johan.gill at agama.tv  Thu Jan  7 10:46:52 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 10:46:52 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
Message-ID: <4B45AD8C.5000405@agama.tv>

Hi devs,
the company where I work has done some work on Python, and the question 
is how this work, owned by the company, can be contributed to the 
community properly. Are there any license issues or other pitfalls we 
need to think about? I imagine that other companies have contributed 
before, so this is probably an already solved problem.

Regards
Johan Gill
Agama Technologies



From regebro at gmail.com  Thu Jan  7 12:12:17 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 12:12:17 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45AD8C.5000405@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
Message-ID: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
> Hi devs,
> the company where I work has done some work on Python, and the question is
> how this work, owned by the company, can be contributed to the community
> properly. Are there any license issues or other pitfalls we need to think
> about? I imagine that other companies have contributed before, so this is
> probably an already solved problem.

I'm not a license lawyer, but typically your company needs to give the
code to the community. Yes, it means it stops owning it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From hodgestar+pythondev at gmail.com  Thu Jan  7 12:28:01 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 7 Jan 2010 13:28:01 +0200
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <fb73205e1001070328j44ac747fu7232a89b559ad0d9@mail.gmail.com>

On Thu, Jan 7, 2010 at 1:12 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

This is incorrect.

The correct information is at http://www.python.org/psf/contrib/.

Schiavo
Simon


From swamy.sangamesh at gmail.com  Thu Jan  7 11:46:59 2010
From: swamy.sangamesh at gmail.com (swamy sangamesh)
Date: Thu, 7 Jan 2010 16:16:59 +0530
Subject: [Python-Dev] test_ctypes failure on AIX 5.3 using python-2.6.2 and
	libffi-3.0.9
Message-ID: <c3369a531001070246s441e159cg35b4cab4e3f65541@mail.gmail.com>

Hi All,

I built the python-2.6.2 with the latest libffi-3.0.9 in AIX 5.3 using xlc
compiler.
When i try to run the ctypes test cases, two failures are seen in
test_bitfields.

*test_ints (ctypes.test.test_bitfields.C_Test) ... FAIL
test_shorts (ctypes.test.test_bitfields.C_Test) ... FAIL*

I have attached the full test case result.

If i change the type c_int and c_short to c_unit and c_ushort of class
"BITS(Structure)" in file
test_bitfields.py then no failures are seen.

Has anyone faced the similar issue or any help is appreciated.


Thanks,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0005.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctype-testcases
Type: application/octet-stream
Size: 22996 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0005.obj>

From ncoghlan at gmail.com  Thu Jan  7 13:23:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 07 Jan 2010 22:23:11 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <4B45D22F.40404@gmail.com>

Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
>> Hi devs,
>> the company where I work has done some work on Python, and the question is
>> how this work, owned by the company, can be contributed to the community
>> properly. Are there any license issues or other pitfalls we need to think
>> about? I imagine that other companies have contributed before, so this is
>> probably an already solved problem.
> 
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

As Simon pointed out, while some organisations do work that way, the PSF
isn't one of them.

The PSF only requires that the code be contributed under a license that
then allows us to turn around and redistribute it under a different open
source license without requesting additional permission from the
copyright holder. For corporate contributions, I believe the contributor
agreement needs to be signed by an authorised agent of the company - the
place to check that would probably be psf at python.org (that's the email
address for the PSF board).

Assuming the subject line relates to the code that you would like to
contribute though, that particular change is unlikely to happen - 2.6 is
in maintenance mode and changing RLock from a Python implementation to
the faster C one is solidly in new feature territory. Although a
backport of the 3.2 C RLock implementation to 2.7 could be useful, I
doubt that backporting code provided by an existing committer would be
the subject of this query :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Thu Jan  7 14:11:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:11:27 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>

On Thu, Jan 7, 2010 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder.

Even if the contributed code as in this case is a method in an
existing file? How does that work, how do they keep ownership of one
method in the threading.py method? :-)

> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)

Ah. I probably misunderstood what the suggested contribution was.
Maybe it was a separate file, which I didn't get.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Thu Jan  7 14:15:09 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 07 Jan 2010 13:15:09 +0000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
Message-ID: <4B45DE5D.3030104@voidspace.org.uk>

On 07/01/2010 13:11, Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 13:23, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>    
>> As Simon pointed out, while some organisations do work that way, the PSF
>> isn't one of them.
>>
>> The PSF only requires that the code be contributed under a license that
>> then allows us to turn around and redistribute it under a different open
>> source license without requesting additional permission from the
>> copyright holder.
>>      
> Even if the contributed code as in this case is a method in an
> existing file? How does that work, how do they keep ownership of one
> method in the threading.py method? :-)
>
>    

When contributing code to Python all work remains copyright the original 
author. The combined work is copyright *everyone*. The PSF has a license 
to distribute it, which is all that is important.

How you meaningfully exercise your ownership over chunks of code is left 
for the reader to determine...

(i.e. copyright and ownership are legal terms that don't necessarily 
mean anything *practical* in these situations.)

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From stefan_ml at behnel.de  Thu Jan  7 14:30:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 14:30:16 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
Message-ID: <hi4nl7$2ej$1@ger.gmane.org>

Guido van Rossum, 07.01.2010 05:29:
> A better rule would be "you may access the memory buffer in a PyString
> or PyUnicode object with the GIL released as long as you own a
> reference to the string object." Everything else is out of bounds (or
> not worth the bother).

Is that a "yes" regarding the OP's original question about releasing the 
GIL during regexp searches?

Stefan



From regebro at gmail.com  Thu Jan  7 14:36:42 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:36:42 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DE5D.3030104@voidspace.org.uk>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
	<4B45DE5D.3030104@voidspace.org.uk>
Message-ID: <319e029f1001070536t49f76899j111212b02c14f192@mail.gmail.com>

On Thu, Jan 7, 2010 at 14:15, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> (i.e. copyright and ownership are legal terms that don't necessarily mean
> anything *practical* in these situations.)

OK, fair enough. :-)
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Thu Jan  7 15:05:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 15:05:23 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <hi4pn2$9so$1@ger.gmane.org>

MRAB, 07.01.2010 04:07:
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER

Py_UNICODE_TOLOWER looks safe to me at first glance.


> or PyErr_SetString?

Certainly not safe.


> Is there an easy way to find out?

Release it and fix any crashes? Note that this isn't a safe solution, 
though, as some GIL requiring code may be platform specific. So a better 
approach might be to extract any obviously problematic stuff from the 
existing code (such as any exception handling, explicit ref-counting or 
object creation), and *then* try to release the GIL.

Stefan



From solipsis at pitrou.net  Thu Jan  7 16:38:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 15:38:31 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <loom.20100107T163459-842@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out?

There is no "easy way" to do so. The only safe way is to examine all the
functions or macros you want to call with the GIL released, and assess whether
it is safe to call them. As already pointed out, no reference count should be
changed, and generally no mutable container should be accessed, except if that
container is known not to be referenced anywhere else (that would be the case
for e.g. a list that your function has created and is busy populating).

I agree that releasing the GIL when doing non-trivial regex searches is a
worthwhile research, so please don't give up immediately :-)

Regards

Antoine Pitrou.




From olemis at gmail.com  Thu Jan  7 19:24:59 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 7 Jan 2010 13:24:59 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
	<24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
Message-ID: <24ea26601001071024m2abd988fuc77f94845d57483a@mail.gmail.com>

On Mon, Jan 4, 2010 at 9:24 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
>> Thanks for the quick response.
>>
>> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> but maybe a
>> discussion could start about a new, less hacky, way of doing the same
>>
>
> I am strongly -1 for modifying the classes in ?traditional? unittest
> module [2]_ (except that I am strongly +1 for the package structure
> WITHOUT TOUCHING anything else ...) , and the more I think about it I
> am more convinced ... but anyway, this not a big deal (and in the end
> what I think is not that relevant either ... :o). So ...
>

IOW, if I had all the freedom to implement it, after the pkg structure
I'd do something like :

{{{
#!python

class TestResult:
    # Everything just the same
    def _is_relevant_tb_level(self, tb):
        return '__unittest' in tb.tb_frame.f_globals

class BetterTestResult(TestResult):
    # Further code ... maybe ;o)
    #
    def _is_relevant_tb_level(self, tb):
        # This or anything else you might want to do ;o)
        #
        globs = tb.tb_frame.f_globals
        is_relevant =  '__name__' in globs and \
            globs["__name__"].startswith("unittest")
        del globs
        return is_relevant
}}}

that's what inheritance is for ;o) ... but quite probably that's not
gonna happen, just a comment .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Ubuntu sustituye GIMP por F-Spot  -
http://feedproxy.google.com/~r/simelo-es/~3/-g48D6T6Ojs/ubuntu-sustituye-gimp-por-f-spot.html


From martin at v.loewis.de  Thu Jan  7 21:24:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:24:32 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi4nl7$2ej$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
	<hi4nl7$2ej$1@ger.gmane.org>
Message-ID: <4B464300.2020204@v.loewis.de>

>> A better rule would be "you may access the memory buffer in a PyString
>> or PyUnicode object with the GIL released as long as you own a
>> reference to the string object." Everything else is out of bounds (or
>> not worth the bother).
> 
> Is that a "yes" regarding the OP's original question about releasing the
> GIL during regexp searches?

No, because the regex engine may also operate on buffers that start
moving around when you release the GIL.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:27:09 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:27:09 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B46439D.9030604@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

I don't think that's possible. The regex engine can also operate on
objects whose representation may move in memory when you don't hold
the GIL (e.g. buffers that get mutated). Even if they stay in place -
if their contents changes, regex results may be confusing.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:31:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:31:21 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
Message-ID: <4B464499.4020305@v.loewis.de>

> I would like to use astgen.py to generate python classes corresponding to the 
> AST of something I have defined in a .asdl file, along the line of what is
> apparently done for the python AST itself. I thought astgen.py would
> take as an argument a .asdl file, but apparently it instead process a file
> called ast.txt. Where does this file come from ? Is it generated from
> Python.asdl ?

astgen.py is not used to process asdl files; ast.txt lives right next to
astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

HTH,
Martin


From foom at fuhm.net  Thu Jan  7 21:36:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 7 Jan 2010 15:36:37 -0500
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B46439D.9030604@v.loewis.de>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>

On Jan 7, 2010, at 3:27 PM, Martin v. L?wis wrote:

>> I've been wondering whether it's possible to release the GIL in the
>> regex engine during matching.
>
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated). Even if they stay in place -
> if their contents changes, regex results may be confusing.

It seems probably worthwhile to optimize for the common case of using  
the regexp engine on an immutable object of type "str" or "bytes", and  
allow releasing the GIL in *that* case, even if you have to keep it  
for the general case.

James

From martin at v.loewis.de  Thu Jan  7 21:45:42 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:45:42 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
Message-ID: <4B4647F6.2060309@v.loewis.de>

>>> I've been wondering whether it's possible to release the GIL in the
>>> regex engine during matching.
>>
>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated). Even if they stay in place -
>> if their contents changes, regex results may be confusing.
> 
> It seems probably worthwhile to optimize for the common case of using
> the regexp engine on an immutable object of type "str" or "bytes", and
> allow releasing the GIL in *that* case, even if you have to keep it for
> the general case.

Right. This problem was the one that I thought of first.

Thinking about these things is fairly difficult (to me, at least), so
I think I could only tell whether I would consider a patch thread-safe
that released the GIL around matching under selected circumstances -
if I had the patch available. I don't see any obvious reason (assuming
Guido's list of conditions holds - i.e. you are holding references to
everything you access).

Regards,
Martin


From ncoghlan at gmail.com  Thu Jan  7 21:48:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 08 Jan 2010 06:48:05 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DECB.7070907@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com> <4B45DECB.7070907@agama.tv>
Message-ID: <4B464885.40806@gmail.com>

Johan Gill wrote:
> Yes, it is the new RLock implementation.
> If I understood this correctly, we should make a patch against trunk if
> anything should be contributed.

Yep.

> Do you mean that we wouldn't need the paperwork for backporting the
> original patch committed to py3k?

Whether or not a contributor agreement was essential for this particular
contribution would depend on how much new code was needed for the
backport, but the bulk of the copyright on the C RLock code would remain
with Antoine regardless.

However, sorting through the legalities of the contributor agreement
really is the best way to make sure every is squared away nice and
neatly from a legal point of view.

After all, even if I was a lawyer (which I'm not, I'm just a developer
with an interest in licensing issues), I still wouldn't be *your* lawyer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Jan  7 21:43:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 20:43:17 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <loom.20100107T214211-130@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated).

Why is it a problem? If we get a buffer through the new buffer API, the object
should ensure that the representation isn't moved away until the buffer is 
released.

Regards

Antoine.




From martin at v.loewis.de  Thu Jan  7 21:59:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:59:29 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B464B31.7040406@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

Ok, here is another problem: SRE_OP_REPEAT uses PyObject_MALLOC,
which requires the GIL (it then also may call PyErr_NoMemory,
which also requires the GIL).

Regards,
Martin


From johan.gill at agama.tv  Thu Jan  7 14:16:59 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 14:16:59 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <4B45DECB.7070907@agama.tv>

On 01/07/2010 01:23 PM, Nick Coghlan wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder. For corporate contributions, I believe the contributor
> agreement needs to be signed by an authorised agent of the company - the
> place to check that would probably be psf at python.org (that's the email
> address for the PSF board).
>
> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)
>
> Regards,
> Nick.
>
>    
Yes, it is the new RLock implementation.
If I understood this correctly, we should make a patch against trunk if 
anything should be contributed.
Do you mean that we wouldn't need the paperwork for backporting the 
original patch committed to py3k?

Regards
Johan Gill
Agama Technologies



From yoann.padioleau at facebook.com  Thu Jan  7 22:07:55 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Thu, 7 Jan 2010 13:07:55 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464499.4020305@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
Message-ID: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>


On Jan 7, 2010, at 12:31 PM, Martin v. L?wis wrote:

>> I would like to use astgen.py to generate python classes corresponding to the 
>> AST of something I have defined in a .asdl file, along the line of what is
>> apparently done for the python AST itself. I thought astgen.py would
>> take as an argument a .asdl file, but apparently it instead process a file
>> called ast.txt. Where does this file come from ? Is it generated from
>> Python.asdl ?
> 
> astgen.py is not used to process asdl files; ast.txt lives right next to
> astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

Yes, I know that. That's why I asked about the relation between ast.txt and Python.adsl.
If internally the parser uses the .adsl, but expose as a reflection mechanism things
that were generated from ast.txt, then there could be a mismatch. Where does ast.txt comes from ? Shouldn't it be generated itself from Python.adsl ?

So we would have

Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py containing all the UnarySub, Expression, classes that represents a Python AST.



> 
> HTH,
> Martin



From martin at v.loewis.de  Thu Jan  7 22:11:36 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 07 Jan 2010 22:11:36 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <loom.20100107T214211-130@post.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
Message-ID: <4B464E08.5020703@v.loewis.de>

>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated).
> 
> Why is it a problem? If we get a buffer through the new buffer API, the object
> should ensure that the representation isn't moved away until the buffer is 
> released.

In 2.7, we currently get the buffer with bf_getreadbuffer. In 3.x, we have

    /* Release the buffer immediately --- possibly dangerous
       but doing something else would require some re-factoring
    */
    PyBuffer_Release(&view);


Even if we do use the new API, and correctly, it still might be
confusing if the contents of the buffer changes underneath.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 22:16:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 22:16:12 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
Message-ID: <4B464F1C.7020404@v.loewis.de>

>> astgen.py is not used to process asdl files; ast.txt lives right
>> next to astgen.py. Instead, the asdl file is processed by
>> Parser/asdl_c.py.
> 
> Yes, I know that. That's why I asked about the relation between
> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
> expose as a reflection mechanism things that were generated from
> ast.txt, then there could be a mismatch. Where does ast.txt comes
> from ? Shouldn't it be generated itself from Python.adsl ?

What you may not be aware of is that Tools/compiler (and the
compiler package that it builds on) are both unused and unmaintained.

If the package stops working correctly - tough luck.

> So we would have
> 
> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
> containing all the UnarySub, Expression, classes that represents a
> Python AST.

No - what actually happens in Python 3.x is this: both the compiler
package and Tools/compiler are removed.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 01:10:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 01:10:35 +0100
Subject: [Python-Dev] Improve open() to support reading file starting with
	an unicode BOM
Message-ID: <201001080110.35800.victor.stinner@haypocalc.com>

Hi,

Builtin open() function is unable to open an UTF-16/32 file starting with a 
BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
file starting with a BOM, read()/readline() returns also the BOM whereas the 
BOM should be "ignored".

See recent issues related to reading an UTF-8 text file including a BOM: #7185 
(csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with 
the UTF-8-SIG encoding, but it's possible to do better.

I propose to improve open() (TextIOWrapper) by using the BOM to choose the 
right encoding. I think that only files opened in read only mode should 
support this new feature. *Read* the BOM in a *write* only file would cause 
unexpected behaviours.

Since my proposition changes the result TextIOWrapper.read()/readline() for 
files starting with a BOM, we might introduce an option to open() to enable 
the new behaviour. But is it really needed to keep the backward compatibility?

I wrote a proof of concept attached to the issue #7651. My patch only changes 
the behaviour of TextIOWrapper for reading files starting with a BOM. It 
doesn't work yet if a seek() is used before the first read.

-- 
Victor Stinner
http://www.haypocalc.com/


From guido at python.org  Fri Jan  8 01:52:20 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 16:52:20 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>

I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
talk. And for the other two, perhaps it would make more sense to have
a separate encoding-guessing function that takes a binary stream and
returns a text stream wrapping it with the proper encoding?

--Guido

On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
> file starting with a BOM, read()/readline() returns also the BOM whereas the
> BOM should be "ignored".
>
> See recent issues related to reading an UTF-8 text file including a BOM: #7185
> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
> the UTF-8-SIG encoding, but it's possible to do better.
>
> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
> right encoding. I think that only files opened in read only mode should
> support this new feature. *Read* the BOM in a *write* only file would cause
> unexpected behaviours.
>
> Since my proposition changes the result TextIOWrapper.read()/readline() for
> files starting with a BOM, we might introduce an option to open() to enable
> the new behaviour. But is it really needed to keep the backward compatibility?
>
> I wrote a proof of concept attached to the issue #7651. My patch only changes
> the behaviour of TextIOWrapper for reading files starting with a BOM. It
> doesn't work yet if a seek() is used before the first read.
>
> --
> Victor Stinner
> http://www.haypocalc.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Fri Jan  8 03:23:08 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 02:23:08 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <4B46970C.9010306@mrabarnett.plus.com>

Guido van Rossum wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
> 
Alternatively, have a universal UTF-8/16/32 encoding, ie one that 
expects UTF-8,
with or without BOM, or UTF-16/32 with BOM.
> 
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
>> See recent issues related to reading an UTF-8 text file including a BOM: #7185
>> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
>> the UTF-8-SIG encoding, but it's possible to do better.
>>
>> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
>> right encoding. I think that only files opened in read only mode should
>> support this new feature. *Read* the BOM in a *write* only file would cause
>> unexpected behaviours.
>>
>> Since my proposition changes the result TextIOWrapper.read()/readline() for
>> files starting with a BOM, we might introduce an option to open() to enable
>> the new behaviour. But is it really needed to keep the backward compatibility?
>>
>> I wrote a proof of concept attached to the issue #7651. My patch only changes
>> the behaviour of TextIOWrapper for reading files starting with a BOM. It
>> doesn't work yet if a seek() is used before the first read.
>>


From nick.bastin at gmail.com  Fri Jan  8 04:11:03 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Thu, 7 Jan 2010 22:11:03 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
Message-ID: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>

I think this problem probably needs to move over to distutils-sig, as
it doesn't seem to be specific to the way that Python itself uses
distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
linux and solaris and automatically adds '.' to the library_dirs, and
I suspect it just needs to do this on FreeBSD as well (adding bsd to
the list of platforms for which this is performed "solves" the
problem, but I don't pretend to know enough about either distutils or
freebsd to determine if this is the correct solution).

--
Nick


From glyph at twistedmatrix.com  Fri Jan  8 04:34:36 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 7 Jan 2010 22:34:36 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>



On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>> 
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".

> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

It *is* crazy, but unfortunately rather common.  Wikipedia has a good description of the issues: <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as being UTF-8, so it's become a convention to do that.  That's not good enough, so you need to guess the encoding as well to make sure, but if there is a BOM and you can otherwise verify that the file is probably UTF-8 encoded, you should discard it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/1bc40870/attachment-0005.htm>

From tseaver at palladion.com  Fri Jan  8 05:17:28 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 Jan 2010 23:17:28 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <hi6bko$d0h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
> linux and solaris and automatically adds '.' to the library_dirs, and
> I suspect it just needs to do this on FreeBSD as well (adding bsd to
> the list of platforms for which this is performed "solves" the
> problem, but I don't pretend to know enough about either distutils or
> freebsd to determine if this is the correct solution).

I wouldn't say it needed discussion on the SIG:  just create a bug
report, with the tentative patch you have worked out, and get it
assigned to Tarek.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGsdQACgkQ+gerLs4ltQ5BMQCgtV8snMXH/6dDwgdN4sIJljLd
koYAoKq6c0tKsRSrITHcygu4Od9FVzF5
=BJaE
-----END PGP SIGNATURE-----



From guido at python.org  Fri Jan  8 05:21:04 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 20:21:04 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
Message-ID: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>

On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>
>
> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
>
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>
> file starting with a BOM, read()/readline() returns also the BOM whereas the
>
> BOM should be "ignored".
>
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>
> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
> description of the issues:
> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
> being UTF-8, so it's become a convention to do that. ?That's not good
> enough, so you need to guess the encoding as well to make sure, but if there
> is a BOM and you can otherwise verify that the file is probably UTF-8
> encoded, you should discard it.

That doesn't make sense. If the file isn't UTF-8 you can't see the
BOM, because the BOM itself is UTF-8-encoded.

(And yes, I know this happens. Doesn't mean we need to auto-guess by
default; there are lots of issues e.g. what should happen after
seeking to offset 0?)

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Fri Jan  8 07:06:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 08 Jan 2010 15:06:16 +0900
Subject: [Python-Dev] Improve open() to support reading file
	starting	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
 > talk.

That doesn't stop many applications from doing it.  Python should
perhaps<wink,nudge> not produce UTF-8 + BOM without a disclaimer of
indemnification against all resulting damage, signed in blood, from
the user for each instance.

But it should do something sane when reading such files.  I can't
really see any harm in throwing it away, especially since use of
ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
IIRC.






From tseaver at palladion.com  Fri Jan  8 07:12:12 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 01:12:12 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <hi6ibr$qag$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>
>> BOM should be "ignored".
>>
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.
> 
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

The BOM should not be seekeable if the file is opened with the proposed
"guess encoding from BOM" mode:  it isn't properly part of the stream at
all in that case.

A UTF-8 BOM is an absurditiy, but it exists *everywhere* in the wild:
Python would do wll to make it as easy as possible to consume such
files, as well as the non-insane versions (UTF-16 / UTF-32 BOMs).  In
the best of all possible worlds, I would just try opening the file so:

  f = open('/path/to/file', 'r', encoding="DWIFM")

and any BOM present would set the encoding for the remainder of the stream..



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGzLsACgkQ+gerLs4ltQ5+cwCdGfycPdj6+cPfD23vH644SpHL
sI0AoLGD7nfgMEJdJhBr90yjQQHfDgcJ
=js+2
-----END PGP SIGNATURE-----



From glyph at twistedmatrix.com  Fri Jan  8 08:55:27 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 8 Jan 2010 02:55:27 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>


On Jan 7, 2010, at 11:21 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> 
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>> 
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>> 
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.  If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal.  It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I think it's pretty clear that the BOM should still be skipped in that case ...



From martin at v.loewis.de  Fri Jan  8 10:05:17 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:05:17 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <4B46F54D.9090103@v.loewis.de>

>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I think what Glyph meant is this: if a file starts with the UTF-8
signature, assume it's UTF-8. Then validate the assumption against the
rest of the file also, and then process it as UTF-8. If the rest clearly
is not UTF-8, assume that the UTF-8 signature is bogus.

I understood this proposal as a general processing guideline, not
something the io library should do (but, say, a text editor).

FWIW, I'm personally in favor of using the UTF-8 signature. If people
consider them crazy talk, that may be because UTF-8 can't possibly have
a byte order - hence I call it a signature, not the BOM. As a signature,
I don't consider it crazy at all. There is a long tradition of having
magic bytes in files (executable files, Postscript, PDF, ... - see
/etc/magic). Having a magic byte sequence for plain text to denote the
encoding is useful and helps reducing moji-bake. This is the reason it's
used on Windows: notepad would normally assume that text is in the ANSI
code page, and for compatibility, it can't stop doing that. So the UTF-8
signature gives them an exit strategy.

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:06:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:06:34 +0100
Subject: [Python-Dev] Improve open() to support reading file	starting
 with an unicode BOM
In-Reply-To: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B46F59A.5060905@v.loewis.de>

> But it should do something sane when reading such files.  I can't
> really see any harm in throwing it away, especially since use of
> ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
> IIRC.

And indeed it does, when you open the file in the utf-8-sig encoding.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 10:08:30 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 10:08:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46970C.9010306@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<4B46970C.9010306@mrabarnett.plus.com>
Message-ID: <201001081008.30904.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 03:23:08, MRAB a ?crit :
> Guido van Rossum wrote:
> > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> > talk. And for the other two, perhaps it would make more sense to have
> > a separate encoding-guessing function that takes a binary stream and
> > returns a text stream wrapping it with the proper encoding?
> 
> Alternatively, have a universal UTF-8/16/32 encoding, ie one that
> expects UTF-8,
> with or without BOM, or UTF-16/32 with BOM.

Do you mean open(filename, encoding="BOM")? I suppose that "BOM" would be a 
magical value specific to read a text file (open(filename, "r")), not a real 
codec?

Otherwise which encoding should be used for open(filename, "w", 
encoding="BOM")?

-- 
Victor Stinner
http://www.haypocalc.com/


From martin at v.loewis.de  Fri Jan  8 10:10:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:10:23 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with	an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <4B46F67F.60604@v.loewis.de>

> Builtin open() function is unable to open an UTF-16/32 file starting with a 
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
> file starting with a BOM, read()/readline() returns also the BOM whereas the 
> BOM should be "ignored".

It depends. If you use the utf-8-sig encoding, it *will* ignore the
UTF-8 signature.

> Since my proposition changes the result TextIOWrapper.read()/readline() for 
> files starting with a BOM, we might introduce an option to open() to enable 
> the new behaviour. But is it really needed to keep the backward compatibility?

Absolutely. And there is no need to produce a new option, but instead
use the existing options: define an encoding that auto-detects the
encoding from the family of BOMs. Maybe you call it encoding="sniff".

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:11:51 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:11:51 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <4B46F6D7.1050301@v.loewis.de>

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.

I'm fairly skeptical that anybody on distutils SIG is interested in
details of the Python build process...

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 11:27:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:27:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <201001081127.44044.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
(...)
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I wrote a new version of my patch (version 3):

 * don't change the default behaviour: use open(filename, encoding="BOM") to 
check the BOM is there is any
 * fix for seek(0): always ignore the BOM
 * add an unit test: check that the right encoding is detect, but also the the 
BOM is ignored (especially after a seek(0))

BOM encoding doesn't work for writing into a file, so open(filename, "w", 
encoding="BOM") raises a ValueError.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:31:37 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:31:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <201001081131.37492.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 01:52:20, Guido van Rossum a ?crit :
> And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

I choosed to modify open()+TextIOWrapper instead of writing a new function 
because I would like to avoid an extra read operation (syscall) on the file. 
With my implementation, no specific read operation is needed to detect the 
BOM. The BOM is simply checked in the first _read_chunk().

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:40:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:40:28 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Improve_open=28=29_to_support_reading?=
 =?iso-8859-1?q?_file_starting_with=09an_unicode_BOM?=
In-Reply-To: <4B46F67F.60604@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
Message-ID: <201001081140.28124.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
> > Builtin open() function is unable to open an UTF-16/32 file starting with
> > a BOM if the encoding is not specified (raise an unicode error). For an
> > UTF-8 file starting with a BOM, read()/readline() returns also the BOM
> > whereas the BOM should be "ignored".
> 
> It depends. If you use the utf-8-sig encoding, it *will* ignore the
> UTF-8 signature.

Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
remove the BOM after the first read (much harder if you use a module like 
ConfigParser or csv).

> > Since my proposition changes the result TextIOWrapper.read()/readline()
> > for files starting with a BOM, we might introduce an option to open() to
> > enable the new behaviour. But is it really needed to keep the backward
> > compatibility?
> 
> Absolutely. And there is no need to produce a new option, but instead
> use the existing options: define an encoding that auto-detects the
> encoding from the family of BOMs. Maybe you call it encoding="sniff".

Good idea, I choosed open(filename, encoding="BOM").

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Fri Jan  8 15:27:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:27:42 +0000 (UTC)
Subject: [Python-Dev] GIL required for _all_ Python calls?
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de>
Message-ID: <hi7fcu$gs7$1@ger.gmane.org>

Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
> 
> Even if we do use the new API, and correctly, it still might be
> confusing if the contents of the buffer changes underneath.

Well, no more confusing than when you compute a SHA1 hash or zlib-
compress the buffer, is it?

Regards

Antoine




From solipsis at pitrou.net  Fri Jan  8 15:34:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:34:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <loom.20100108T153305-228@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any

Well, I think if we implement this the default behaviour *should* be changed.
It looks a bit senseless to have two different "auto-choose" options, one with
encoding=None and one with encoding="BOM".

Regards

Antoine.




From guido at python.org  Fri Jan  8 16:48:44 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:48:44 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi7fcu$gs7$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de> 
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de> <hi7fcu$gs7$1@ger.gmane.org>
Message-ID: <ca471dc21001080748lbc18bbx4c4ec2d3f4f027e@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
>>
>> Even if we do use the new API, and correctly, it still might be
>> confusing if the contents of the buffer changes underneath.
>
> Well, no more confusing than when you compute a SHA1 hash or zlib-
> compress the buffer, is it?

That depends. Algorithms that make exactly one pass over the buffer
will run fine (maybe producing a meaningless result). But the regex
matcher may scan the buffer repeatedly (for backtracking purposes) and
it would take a considerable analysis to prove that cannot mess up its
internal data structures if the data underneath changes. (I give it a
decent chance that it's fine, but since it was written without ever
considering this possibility I'm not 100% sure.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:52:48 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:52:48 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
Message-ID: <ca471dc21001080752r312dd47fx32486a95336160ec@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:55 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
> I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.

And I'm saying that it is, with as much certainty as we can ever guess
the encoding of a file.

> If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal. ?It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

Sure, a Latin-1-encoded file could start with the same pattern that is
a UTF-8-encoded BOM. But at that point, a UTF-16-encoded file is also
valid Latin-1.

The question was in the context of encoding-guessing; if we're
guessing, a UTF-8-encoded BOM cannot signify anything else but UTF-8.
(Ditto for UTF-16 and UTF-32 BOMs.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:54:15 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:54:15 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100108T153305-228@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
Message-ID: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>
>> I wrote a new version of my patch (version 3):
>>
>> ?* don't change the default behaviour: use open(filename, encoding="BOM") to
>> check the BOM is there is any
>
> Well, I think if we implement this the default behaviour *should* be changed.
> It looks a bit senseless to have two different "auto-choose" options, one with
> encoding=None and one with encoding="BOM".

Well there *are* two different auto options: use the environment
variables (LANG etc.) or inspect the contents of the file. I think it
would be useful to have ways to specify both.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:56:46 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:56:46 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <ca471dc21001080756p5cb9f560x2a4443efa441fa7b@mail.gmail.com>

On Fri, Jan 8, 2010 at 1:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that. ?That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>>
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
>
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.
>
> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
>
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Sure. I said "crazy talk" only to stir up discussion. Which worked. :-)

Also, I don't want Python's default behavior to change -- sniffing the
encoding should be a separate option.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:59:45 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:59:45 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi6ibr$qag$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<hi6ibr$qag$1@ger.gmane.org>
Message-ID: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
> The BOM should not be seekeable if the file is opened with the proposed
> "guess encoding from BOM" mode: ?it isn't properly part of the stream at
> all in that case.

This feels about right to me. There are still questions though:
immediately after opening a file with a BOM, what should .tell()
return? And regardless of that, .seek(0) should put the file in that
same initial state.

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Fri Jan  8 17:03:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:03:07 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <loom.20100108T170053-283@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.




From mal at egenix.com  Fri Jan  8 17:25:22 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 17:25:22 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <4B475C72.1010207@egenix.com>

Guido van Rossum wrote:
> On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>>
>>> I wrote a new version of my patch (version 3):
>>>
>>>  * don't change the default behaviour: use open(filename, encoding="BOM") to
>>> check the BOM is there is any
>>
>> Well, I think if we implement this the default behaviour *should* be changed.
>> It looks a bit senseless to have two different "auto-choose" options, one with
>> encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Shouldn't this encoding guessing be a separate function that you call
on either a file or a seekable stream ?

After all, detecting encodings is just as useful to have for non-file
streams. You'd then avoid having to stuff everything into
a single function call and also open up the door for more complex
application specific guess work or defaults.

The whole process would then have two steps:

 1. guess encoding

  import codecs
  encoding = codecs.guess_file_encoding(filename)

 2. open the file with the found encoding

  f = open(filename, encoding=encoding)

For seekable streams f, you'd have:

 1. guess encoding

  import codecs
  encoding = codecs.guess_stream_encoding(f)

 2. wrap the stream with a reader for the found encoding

  reader_class = codecs.getreader(encoding)
  g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Jan  8 17:31:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:31:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <loom.20100108T171644-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver <at> palladion.com>
wrote:
> > The BOM should not be seekeable if the file is opened with the proposed
> > "guess encoding from BOM" mode:  it isn't properly part of the stream at
> > all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return?

tell() in the context of text I/O is specified to return an "opaque cookie". So
whatever value it returns would probably be fine, as long as seeking to that
value leaves the file in an acceptable state.

Rewinding (seeking to 0) in the presence of a BOM is already reasonably
supported by the TextIOWrapper object:

>>> dec = codecs.getincrementaldecoder('utf-16')()
>>> dec.decode(b'\xff\xfea\x00b\x00')
'ab'
>>> dec.decode(b'\xff\xfea\x00b\x00')
'\ufeffab'
>>> 
>>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
>>> f = io.TextIOWrapper(bio, encoding='utf-16')
>>> f.read()
'ab'
>>> f.seek(0)
0
>>> f.read()
'ab'

There are tests for this in test_io.py (test_encoded_writes, line 1929, and
test_append_bom and test_seek_bom, line 2045).

Regards

Antoine.




From python at mrabarnett.plus.com  Fri Jan  8 17:47:18 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 16:47:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081127.44044.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <4B476196.4080409@mrabarnett.plus.com>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
> (...)
>> (And yes, I know this happens. Doesn't mean we need to auto-guess by
>> default; there are lots of issues e.g. what should happen after
>> seeking to offset 0?)
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any
>  * fix for seek(0): always ignore the BOM
>  * add an unit test: check that the right encoding is detect, but also the the 
> BOM is ignored (especially after a seek(0))
> 
> BOM encoding doesn't work for writing into a file, so open(filename, "w", 
> encoding="BOM") raises a ValueError.
> 
I think it's similar to universal newline mode. You can tell it that
you're reading UTF-something-encoded text (common forms only).

The preference is UTF-8, but it could be UTF-8-sig (from Windows), or
possibly UTF-16/32, which really need a BOM because there are multiple
bytes per codepoint, so the bytes could be big-endian or little-endian.

The BOM (or signature) tells you what the encoding is, defaulting to
UTF-8 if there's none. If it subsequently raises a DecodeError, then
so be it!

Maybe there should also be a way of determining what encoding it decided
it was, so that you can then write a new file in that same encoding.


From status at bugs.python.org  Fri Jan  8 18:07:13 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jan 2010 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100108170714.014B078669@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/01/10 - 01/08/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2544 open (+27) / 16937 closed (+15) / 19481 total (+42)

Open issues with patches:  1017

Average duration of open issues: 708 days.
Median duration of open issues: 464 days.

Open Issues Breakdown
   open  2509 (+27)
pending    34 ( +0)

Issues Created Or Reopened (43)
_______________________________

Extended slicing with classic class behaves strangely            01/07/10
       http://bugs.python.org/issue7532    reopened mark.dickinson                
       patch                                                                   

optparse library documentation has an insignificant formatting i 01/01/10
CLOSED http://bugs.python.org/issue7618    created  vazovsky                      
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6           01/01/10
CLOSED http://bugs.python.org/issue7619    created  djc                           
                                                                               

Vim syntax highlight                                             01/02/10
       http://bugs.python.org/issue7620    created  july                          
       patch                                                                   

Test issue                                                       01/02/10
CLOSED http://bugs.python.org/issue7621    created  georg.brandl                  
                                                                               

[patch] improve unicode methods: split() rsplit() and replace()  01/03/10
       http://bugs.python.org/issue7622    created  flox                          
       patch                                                                   

PropertyType missing in Lib/types.py                             01/03/10
CLOSED http://bugs.python.org/issue7623    created  wplappert                     
                                                                               

isinstance(... ,collections.Callable) fails with oldstyle class  01/03/10
       http://bugs.python.org/issue7624    created  rgammans                      
                                                                               

bytearray needs more tests for "b.some_method()[0] is not b"     01/03/10
       http://bugs.python.org/issue7625    created  flox                          
       patch                                                                   

Entity references without semicolon in HTMLParser                01/03/10
CLOSED http://bugs.python.org/issue7626    created  stefan.schweizer              
                                                                               

mailbox.MH.remove() lock handling is broken                      01/04/10
       http://bugs.python.org/issue7627    created  sraustein                     
                                                                               

round() doesn't work correctly in 3.1.1                          01/04/10
CLOSED http://bugs.python.org/issue7628    created  bkovt                         
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs" 01/04/10
CLOSED http://bugs.python.org/issue7629    created  popelkopp                     
                                                                               

Strange behaviour of decimal.Decimal                             01/04/10
CLOSED http://bugs.python.org/issue7630    created  parmax                        
                                                                               

undefined label: bltin-file-objects                              01/04/10
CLOSED http://bugs.python.org/issue7631    created  ezio.melotti                  
                                                                               

dtoa.c: oversize b in quorem                                     01/04/10
       http://bugs.python.org/issue7632    created  skrah                         
                                                                               

decimal.py: type conversion in context methods                   01/04/10
       http://bugs.python.org/issue7633    created  skrah                         
       patch, easy                                                             

next/previous links in documentation skip some sections          01/05/10
CLOSED http://bugs.python.org/issue7634    created  gagenellina                   
                                                                               

19.6 xml.dom.pulldom doc: stub?                                  01/05/10
       http://bugs.python.org/issue7635    created  tjreedy                       
                                                                               

Add a set update action to optparse                              01/05/10
CLOSED http://bugs.python.org/issue7636    created  hardkrash                     
       patch                                                                   

Improve 19.5. xml.dom.minidom doc                                01/05/10
       http://bugs.python.org/issue7637    created  tjreedy                       
                                                                               

Counterintuitive str.splitlines() inconsistency.                 01/05/10
CLOSED http://bugs.python.org/issue7638    created  vencabot_teppoo               
                                                                               

bdist_msi fails on files with long names                         01/05/10
       http://bugs.python.org/issue7639    created  mmm77                         
                                                                               

buffered io seek() buggy                                         01/05/10
       http://bugs.python.org/issue7640    created  pakal                         
       patch                                                                   

Built-in Formatter accepts undocumented presentation type        01/06/10
CLOSED http://bugs.python.org/issue7641    created  lrekucki                      
                                                                               

[patch] Minor improvement in os.system doc                       01/06/10
       http://bugs.python.org/issue7642    created  Val                           
       patch                                                                   

What is an ASCII linebreak?                                      01/06/10
       http://bugs.python.org/issue7643    created  flox                          
                                                                               

bug in nntplib.body() method with possible fix                   01/06/10
       http://bugs.python.org/issue7644    created  mdmullins                     
       easy                                                                    

test_distutils fails on Windows XP                               01/06/10
       http://bugs.python.org/issue7645    created  austin987                     
                                                                               

test_telnetlib fails in Windows XP                               01/06/10
       http://bugs.python.org/issue7646    created  austin987                     
                                                                               

Add statvfs flags to the posix module                            01/06/10
       http://bugs.python.org/issue7647    created  ajax at redhat.com               
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                 01/06/10
       http://bugs.python.org/issue7648    created  austin987                     
       easy                                                                    

"u'%c' % char" broken for chars in range '\x80'-'\xFF'           01/07/10
       http://bugs.python.org/issue7649    created  ezio.melotti                  
       patch, easy                                                             

test_uuid is invalid                                             01/07/10
       http://bugs.python.org/issue7650    created  austin987                     
                                                                               

Python3: guess text file charset using the BOM                   01/07/10
       http://bugs.python.org/issue7651    created  haypo                         
       patch                                                                   

Merge C version of decimal into py3k.                            01/07/10
       http://bugs.python.org/issue7652    created  mark.dickinson                
                                                                               

Fix description of the way the PythonPath Windows registry key w 01/07/10
CLOSED http://bugs.python.org/issue7653    created  BoarGules                     
       patch, needs review                                                     

[patch] Enable additional bytes and memoryview tests.            01/07/10
       http://bugs.python.org/issue7654    created  flox                          
       patch                                                                   

PEP 374 Title usage & script redirection typo fixes              01/07/10
CLOSED http://bugs.python.org/issue7655    created  bab9e9                        
       patch                                                                   

test_hashlib fails on some installations (specifically Neal's re 01/08/10
       http://bugs.python.org/issue7656    created  r.david.murray                
                                                                               

test_ctypes failure on AIX 5.3                                   01/08/10
       http://bugs.python.org/issue7657    created  mallayya                      
       patch                                                                   

OS X pythonw.c compile error with 10.4 or earlier deployment tar 01/08/10
       http://bugs.python.org/issue7658    created  ned.deily                     
                                                                               

Problems with attribute assignment on object instances           01/08/10
       http://bugs.python.org/issue7659    created  pakal                         
                                                                               



Issues Now Closed (40)
______________________

shutil fails when copying to NTFS in Linux                        762 days
       http://bugs.python.org/issue1545    benjamin.peterson             
       patch                                                                   

Test                                                              751 days
       http://bugs.python.org/issue1619    georg.brandl                  
                                                                               

Windows Registry issue with 2.5.2 AMD64 msi                       645 days
       http://bugs.python.org/issue2539    loewis                        
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     618 days
       http://bugs.python.org/issue2698    Daniel26                      
                                                                               

optparse print_usage(),.. methods are not documented              542 days
       http://bugs.python.org/issue3340    ezio.melotti                  
                                                                               

reference leaks in test_distutils                                 500 days
       http://bugs.python.org/issue3660    ezio.melotti                  
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                          20 days
       http://bugs.python.org/issue3745    lemburg                       
       26backport                                                              

Add Option to Bind to a Local IP Address in httplib.py            464 days
       http://bugs.python.org/issue3972    gregory.p.smith               
       patch, easy                                                             

no reference for optparse methods                                 388 days
       http://bugs.python.org/issue4635    ezio.melotti                  
                                                                               

PyArg_Parse* should raise TypeError for float parsed with intege  339 days
       http://bugs.python.org/issue5080    mark.dickinson                
       patch                                                                   

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()              282 days
       http://bugs.python.org/issue5576    mark.dickinson                
       patch                                                                   

PyFrame_GetLineNumber                                             244 days
       http://bugs.python.org/issue5954    georg.brandl                  
       patch                                                                   

PyCode_NewEmpty                                                   244 days
       http://bugs.python.org/issue5959    georg.brandl                  
       patch                                                                   

Add non-command help topics to help completion of cmd.Cmd         241 days
       http://bugs.python.org/issue5991    georg.brandl                  
       patch                                                                   

make error                                                        231 days
       http://bugs.python.org/issue6067    georg.brandl                  
                                                                               

no longer possible to hash arrays                                 229 days
       http://bugs.python.org/issue6071    benjamin.peterson             
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           173 days
       http://bugs.python.org/issue6511    r.david.murray                
                                                                               

use different mechanism for pythonw on osx                        127 days
       http://bugs.python.org/issue6834    ned.deily                     
       easy                                                                    

Py3k doc: "from __future__ import division" not necessary          32 days
       http://bugs.python.org/issue7432    ezio.melotti                  
                                                                               

cPickle: stack underflow in load_pop()                             31 days
       http://bugs.python.org/issue7455    pitrou                        
       patch                                                                   

crash in str.rfind() with an invalid start value                   25 days
       http://bugs.python.org/issue7458    pitrou                        
       patch                                                                   

Implement fastsearch algorithm for rfind/rindex                    26 days
       http://bugs.python.org/issue7462    ezio.melotti                  
       patch                                                                   

GZipFile.readline too slow                                         20 days
       http://bugs.python.org/issue7471    pitrou                        
       patch                                                                   

ssl module documentation: SSLSocket.unwrap description shown twi    5 days
       http://bugs.python.org/issue7592    georg.brandl                  
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            7 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

optparse library documentation has an insignificant formatting i    2 days
       http://bugs.python.org/issue7618    ezio.melotti                  
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6              1 days
       http://bugs.python.org/issue7619    djc                           
                                                                               

Test issue                                                          0 days
       http://bugs.python.org/issue7621    ezio.melotti                  
                                                                               

PropertyType missing in Lib/types.py                                0 days
       http://bugs.python.org/issue7623    benjamin.peterson             
                                                                               

Entity references without semicolon in HTMLParser                   2 days
       http://bugs.python.org/issue7626    r.david.murray                
                                                                               

round() doesn't work correctly in 3.1.1                             0 days
       http://bugs.python.org/issue7628    benjamin.peterson             
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs"    0 days
       http://bugs.python.org/issue7629    r.david.murray                
                                                                               

Strange behaviour of decimal.Decimal                                0 days
       http://bugs.python.org/issue7630    mark.dickinson                
                                                                               

undefined label: bltin-file-objects                                 0 days
       http://bugs.python.org/issue7631    pitrou                        
                                                                               

next/previous links in documentation skip some sections             0 days
       http://bugs.python.org/issue7634    georg.brandl                  
                                                                               

Add a set update action to optparse                                 3 days
       http://bugs.python.org/issue7636    r.david.murray                
       patch                                                                   

Counterintuitive str.splitlines() inconsistency.                    0 days
       http://bugs.python.org/issue7638    flox                          
                                                                               

Built-in Formatter accepts undocumented presentation type           0 days
       http://bugs.python.org/issue7641    eric.smith                    
                                                                               

Fix description of the way the PythonPath Windows registry key w    0 days
       http://bugs.python.org/issue7653    r.david.murray                
       patch, needs review                                                     

PEP 374 Title usage & script redirection typo fixes                 0 days
       http://bugs.python.org/issue7655    georg.brandl                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 22 [patch] improve unicode methods: split() rsplit() and replace()    5 days
open    http://bugs.python.org/issue7622   

 14 Test suite emits many DeprecationWarnings when -3 is enabled      91 days
open    http://bugs.python.org/issue7092   

 13 unicode_escape codec does not escape quotes                        8 days
open    http://bugs.python.org/issue7615   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    0 days
open    http://bugs.python.org/issue7658   

  8 Merge C version of decimal into py3k.                              1 days
open    http://bugs.python.org/issue7652   

  8 "u'%c' % char" broken for chars in range '\x80'-'\xFF'             2 days
open    http://bugs.python.org/issue7649   

  8 decimal.py: type conversion in context methods                     4 days
open    http://bugs.python.org/issue7633   

  8 Patch for [ 735515 ] urllib2 should cache 301 redir              906 days
open    http://bugs.python.org/issue1755841

  7 Test issue                                                         0 days
closed  http://bugs.python.org/issue7621   

  7 Cannot use both read and readline method in same ZipExtFile obj    8 days
open    http://bugs.python.org/issue7610   




From tseaver at palladion.com  Fri Jan  8 22:09:54 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:09:54 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <hi86v3$3j5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> The BOM should not be seekeable if the file is opened with the proposed
>> "guess encoding from BOM" mode:  it isn't properly part of the stream at
>> all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return? And regardless of that, .seek(0) should put the file in that
> same initial state.

I think the behavior should be something like:

 >>> f = open('/path/to/maybe-BOM-encoded-file', 'r', encoding='BOM')
 >>> f.tell()
 0L
 >>> f.seek(-1)
 >>> f.tell() # count of unicode chars in decoded stream
 45L
 >>> f.seek(0)
 >>> f.read(1) # read first unicode char decoded from stream.
 'A'

In other words, the BOM is not readable / seekable at all:  it is
invisible to the consumer of the decoded stream.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHnyIACgkQ+gerLs4ltQ6s3QCgznD+7FbUzfCbe5TS6OcoXjMg
rdgAoJAMEXe2xwLCIwJaZ6XA6rVyTIAi
=oXb3
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:19:10 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:19:10 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B475C72.1010207@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com>
Message-ID: <hi87gg$8ge$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:

> Shouldn't this encoding guessing be a separate function that you call
> on either a file or a seekable stream ?
> 
> After all, detecting encodings is just as useful to have for non-file
> streams.

Other stream sources typically have out-of-band ways to signal the
encoding:  only when reading from the filesystem do we pretty much
*have* to guess, and in that case the BOM / signature is the best
heuristic we have.  Also, some non-file streams are not seekable, and so
can't be guessed via a pre-pass.

> You'd then avoid having to stuff everything into
> a single function call and also open up the door for more complex
> application specific guess work or defaults.
> 
> The whole process would then have two steps:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_file_encoding(filename)

Filename is not enough information:  or do you mean that API to actually
open the stream?

>  2. open the file with the found encoding
> 
>   f = open(filename, encoding=encoding)
> 
> For seekable streams f, you'd have:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_stream_encoding(f)
> 
>  2. wrap the stream with a reader for the found encoding
> 
>   reader_class = codecs.getreader(encoding)
>   g = reader_class(f)
> 


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoU4ACgkQ+gerLs4ltQ5o3QCeLOJ7J91E+5f66vhgu1BUhYh4
9UgAnR2IeCd0BCsPez8ZilGNHJfhRn3Y
=SoPb
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:14:59 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:14:59 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <hi878l$7os$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:

>>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that.  That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
> 
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.

If the programmer opens the file using a "guess using the BOM" encoding,
 Python should *not* attempt to verify that the file is properly
encoded:  it should check for (and consume) any BOM, and then return a
stream which uses the encoding inferred from the BOM.  Any errors should
be handled later, when characters are read, exactly as if the file had
been opened with the same encoding guessed from the BOM.

> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
> 
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Agreed.  Having that marker at the start of the file makes interop with
other tools *much* easier.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoFMACgkQ+gerLs4ltQ73dACffwUfyh6Q9vUnKYf367QFjNcU
RRMAoNuKCWEx7j+MSdTv+UjhAPynBc14
=uAX6
-----END PGP SIGNATURE-----



From eric at trueblade.com  Fri Jan  8 22:40:47 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 8 Jan 2010 16:40:47 -0500 (EST)
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
Message-ID: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>

>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
>>
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
>
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

But what if the file were in (for example) a zip file? I think you
definitely want to have access to this functionality outside of open().

Eric.



From foom at fuhm.net  Fri Jan  8 22:49:23 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 8 Jan 2010 16:49:23 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de> <hi878l$7os$1@ger.gmane.org>
Message-ID: <D481E750-3EE7-4498-9959-B4E34E02FFC6@fuhm.net>

On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote:
>> I understood this proposal as a general processing guideline, not
>> something the io library should do (but, say, a text editor).
>>
>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly  
>> have
>> a byte order - hence I call it a signature, not the BOM. As a  
>> signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote  
>> the
>> encoding is useful and helps reducing moji-bake. This is the reason  
>> it's
>> used on Windows: notepad would normally assume that text is in the  
>> ANSI
>> code page, and for compatibility, it can't stop doing that. So the  
>> UTF-8
>> signature gives them an exit strategy.
>
> Agreed.  Having that marker at the start of the file makes interop  
> with
> other tools *much* easier.

Putting the BOM at the beginning of UTF-8 text files is not a good  
idea, it makes interop much *worse* on a unix system, not better.  
Without the BOM, most commands do the right thing with UTF-8 text.  
E.g. to concatenate two files:

$ cat file-1 file-2 > file-3

With a BOM at the beginning of the file, it won't work right. Of  
course, you could modify "cat" (and every other stream processing  
command) to know how to consume and emit BOMs, and omit the extra one  
that would show up in the middle of the stream...but even that can't  
work; what about:
$ (cat file-1; cat file-2) > file-3.

Should the shell now know that when you run multiple commands, it  
should eat the BOM emitted from the second command?

Basically, using a BOM in a utf-8 file is just not a good idea: it  
completely ruins interop with every standard unix tool.

This is not to say that Python shouldn't have a way to read a file  
with a UTF-8 BOM: it just shouldn't encourage you to *write* such files.

James


From mal at egenix.com  Fri Jan  8 22:51:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 22:51:26 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>	<4B475C72.1010207@egenix.com>
	<hi87gg$8ge$1@ger.gmane.org>
Message-ID: <4B47A8DE.1080401@egenix.com>

Tres Seaver wrote:
> M.-A. Lemburg wrote:
> 
>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
> 
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
> 
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

Sure there are non-seekable file streams, but at least when
using StringIO-type streams you don't have that restriction.

An encoding detection function would provide more use in other
cases as well, so instead of hiding away the functionality in
the open() constructor, I'm suggesting to make expose it via
the codecs module.

Applications can then use it where necessary and also provide their
own defaults, using other heuristics.

>> You'd then avoid having to stuff everything into
>> a single function call and also open up the door for more complex
>> application specific guess work or defaults.
> 
>> The whole process would then have two steps:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_file_encoding(filename)
> 
> Filename is not enough information:  or do you mean that API to actually
> open the stream?

Yes. The API would open the file, guess the encoding and then
close it again. If you don't want that to happen, you could use
the second API I mentioned below on the already open file.

Note that this function could detect a lot more encodings with
reasonably high probability than just BOM-prefixed ones,
e.g. we could also add support to detect encoding declarations
such as the ones we use in Python source files.

>>  2. open the file with the found encoding
> 
>>   f = open(filename, encoding=encoding)
> 
>> For seekable streams f, you'd have:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_stream_encoding(f)

I forgot to mention: This API needs to position the file pointer
to the start of the first data byte.

>>  2. wrap the stream with a reader for the found encoding
> 
>>   reader_class = codecs.getreader(encoding)
>>   g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From tseaver at palladion.com  Fri Jan  8 22:59:04 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:59:04 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <hi89r8$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
>>> Shouldn't this encoding guessing be a separate function that you call
>>> on either a file or a seekable stream ?
>>>
>>> After all, detecting encodings is just as useful to have for non-file
>>> streams.
>> Other stream sources typically have out-of-band ways to signal the
>> encoding:  only when reading from the filesystem do we pretty much
>> *have* to guess, and in that case the BOM / signature is the best
>> heuristic we have.  Also, some non-file streams are not seekable, and so
>> can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

If the application expects a possibly-BOM-signature-marked file, but you
pass it mismatched garbage:

  >>> f = open('some.zip', encoding='BOM")

the error handling should be the same as if you passed any other
mismatched encoding:

  >>> f = open('some.zip', encoding='UTF8')

i.e., you discover the error when you try to read from the (non)encoded
stream, not when you open it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHqpwACgkQ+gerLs4ltQ7uAACeKEc+WT4TASGcVl1Hfqe6L9La
I6EAn1pJtngtLWPdothGbYB+zUabEvTW
=TjBK
-----END PGP SIGNATURE-----



From victor.stinner at haypocalc.com  Fri Jan  8 23:10:32 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 23:10:32 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <201001082310.33029.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 22:40:47, Eric Smith a ?crit :
> >> Shouldn't this encoding guessing be a separate function that you call
> >> on either a file or a seekable stream ?
> >>
> >> After all, detecting encodings is just as useful to have for non-file
> >> streams.
> >
> > Other stream sources typically have out-of-band ways to signal the
> > encoding:  only when reading from the filesystem do we pretty much
> > *have* to guess, and in that case the BOM / signature is the best
> > heuristic we have.  Also, some non-file streams are not seekable, and so
> > can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

FYI my patch (encoding="BOM") is implemented in TextIOWrapper, and 
TextIOWrapper takes a binary stream as input, not a filename.

-- 
Victor Stinner
http://www.haypocalc.com/


From yoann.padioleau at facebook.com  Fri Jan  8 23:59:52 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Fri, 8 Jan 2010 14:59:52 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464F1C.7020404@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
	<4B464F1C.7020404@v.loewis.de>
Message-ID: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>


On Jan 7, 2010, at 1:16 PM, Martin v. L?wis wrote:

>>> astgen.py is not used to process asdl files; ast.txt lives right
>>> next to astgen.py. Instead, the asdl file is processed by
>>> Parser/asdl_c.py.
>> 
>> Yes, I know that. That's why I asked about the relation between
>> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
>> expose as a reflection mechanism things that were generated from
>> ast.txt, then there could be a mismatch. Where does ast.txt comes
>> from ? Shouldn't it be generated itself from Python.adsl ?
> 
> What you may not be aware of is that Tools/compiler (and the
> compiler package that it builds on) are both unused and unmaintained.

I see. So if people want to analyze python code they have to use
other tools (like rope?) ? or use reflection ?

> 
> If the package stops working correctly - tough luck.
> 
>> So we would have
>> 
>> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
>> containing all the UnarySub, Expression, classes that represents a
>> Python AST.
> 
> No - what actually happens in Python 3.x is this: both the compiler
> package and Tools/compiler are removed.

Ok. I will then create my own ast classes generator.

Thanks.


> 
> Regards,
> Martin



From g.brandl at gmx.net  Sat Jan  9 00:10:24 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 09 Jan 2010 00:10:24 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<4B46F54D.9090103@v.loewis.de>
	<hi878l$7os$1@ger.gmane.org>
Message-ID: <hi8e1n$sos$1@ger.gmane.org>

Am 08.01.2010 22:14, schrieb Tres Seaver:

>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly have
>> a byte order - hence I call it a signature, not the BOM. As a signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote the
>> encoding is useful and helps reducing moji-bake. This is the reason it's
>> used on Windows: notepad would normally assume that text is in the ANSI
>> code page, and for compatibility, it can't stop doing that. So the UTF-8
>> signature gives them an exit strategy.
> 
> Agreed.  Having that marker at the start of the file makes interop with
> other tools *much* easier.

Except if only 50% of the other tools support the signature.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From martin at v.loewis.de  Sat Jan  9 00:57:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 00:57:46 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>	<4B464499.4020305@v.loewis.de>	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>	<4B464F1C.7020404@v.loewis.de>
	<F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
Message-ID: <4B47C67A.3020302@v.loewis.de>

> I see. So if people want to analyze python code they have to use
> other tools (like rope?) ? or use reflection ?

Correct. One such tool might be the true Python compiler, along
with the _ast module.

Regards,
Martin


From victor.stinner at haypocalc.com  Sat Jan  9 00:59:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 00:59:00 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
Message-ID: <201001090059.00848.victor.stinner@haypocalc.com>

Hi,

Thanks for all the answers! I will try to sum up all ideas here.


(1) Change default open() behaviour or make it optional?

Guido would like to add an option and keep open() unchanged. He wrote that 
checking for BOM and using system locale are too much different to be the same 
option (encoding=None).

Antoine would like to check BOM by default, because both options (system 
locale vs checking for BOM) is the same thing.

About Antoine choice (encoding=None): which file modes would check for a BOM? 
I would like to answer only the read only mode, but then open(filename, "r") 
and open(filename, "r+") would behave differently?

  => 1 point for Guido (encoding="BOM" is more explicit)

Antoine choice has the advantage of directly support UTF-8+BOM, UTF-16 and 
UTF-32 for all applications and all modules using open(filename).

  => 1 point for Antoine


(2) Check for a BOM while reading or detect it before?

Everybody agree that checking BOM is an interesting option and should not be 
limited to open().

Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
name or a binary file-like object: it returns the encoding and seek to the 
file start or just after the BOM.

I dislike this function because it requires extra file operations (open 
(optional), read() and seek()) and it doesn't work if the file is not seekable 
(eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
avoid extra file operations.

Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
any file-like object.


(3) tell() and seek() on a text file starting with a BOM

To be consistent with Antoine example:

   >>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
   >>> f = io.TextIOWrapper(bio, encoding='utf-16')
   >>> f.read()
   'ab'
   >>> f.seek(0)
   0
   >>> f.read()
   'ab'

TextIOWrapper:

 * tell() should return zero at file start,
 * seek(0) should go be to file start,
 * and the BOM should always be "ignored".

I mean:

  with open("utf8bom.txt", encoding="BOM") as fp:
     assert fp.tell() == 0   
     text = fp.read() # no BOM here
     fp.seek(0)
     assert fp.read() == text

--

About my patch:

 - BOM check is explicit: open(filebame,  encoding="BOM")
 - tell() / seek(0) works as expected
 - BOM bytes are always skipped in read() / readlines() result

Hum, I don't know if this email can be called a sum up ;-)

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 01:09:48 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 00:09:48 +0000 (UTC)
Subject: [Python-Dev] Quick sum up about open() + BOM
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <loom.20100109T010657-648@post.gmane.org>

Hello Victor,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> (1) Change default open() behaviour or make it optional?
> 
[...]
> 
> Antoine would like to check BOM by default, because both options (system 
> locale vs checking for BOM) is the same thing.

To be clear, I am not saying it is the same thing. What I think is that it would
be a mistake to use a mildly unreliable heuristic by default (the locale +
device encoding heuristic) but refuse to trust a more reliable heuristic (the
BOM-based detection algorithm).

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Sat Jan  9 01:14:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 00:14:39 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <loom.20100109T010657-648@post.gmane.org>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<loom.20100109T010657-648@post.gmane.org>
Message-ID: <4B47CA6F.5000607@voidspace.org.uk>

On 09/01/2010 00:09, Antoine Pitrou wrote:
> Hello Victor,
>
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>    
>> (1) Change default open() behaviour or make it optional?
>>
>>      
> [...]
>    
>> Antoine would like to check BOM by default, because both options (system
>> locale vs checking for BOM) is the same thing.
>>      
> To be clear, I am not saying it is the same thing. What I think is that it would
> be a mistake to use a mildly unreliable heuristic by default (the locale +
> device encoding heuristic) but refuse to trust a more reliable heuristic (the
> BOM-based detection algorithm).
>    

I concur. On Windows both UTF-8 and signature are very common, yet the 
platform default is the truly awful CP1252.

All the best,

Michael
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From floris.bruynooghe at gmail.com  Sat Jan  9 01:26:05 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 9 Jan 2010 00:26:05 +0000
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B46F6D7.1050301@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
	<4B46F6D7.1050301@v.loewis.de>
Message-ID: <20100109002605.GB13536@laurie.devork>

On Fri, Jan 08, 2010 at 10:11:51AM +0100, "Martin v. L?wis" wrote:
> Nicholas Bastin wrote:
> > I think this problem probably needs to move over to distutils-sig, as
> > it doesn't seem to be specific to the way that Python itself uses
> > distutils.
> 
> I'm fairly skeptical that anybody on distutils SIG is interested in
> details of the Python build process...

Uh, hum.  Unfounded skepticism.  ;-)
But as said filing a bug sounds better in this case.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From v+python at g.nevcal.com  Sat Jan  9 01:47:38 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 16:47:38 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B47D22A.8070305@g.nevcal.com>

On approximately 1/8/2010 3:59 PM, came the following characters from 
the keyboard of Victor Stinner:
> Hi,
>
> Thanks for all the answers! I will try to sum up all ideas here.

One concern I have with this implementation encoding="BOM" is that if 
there is no BOM it assumes UTF-8.  That is probably a good assumption in 
some circumstances, but not in others.

* It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
(cases where the endianness is unspecified) contain a BOM.  Hence, it 
might be that someone would expect a UTF-16LE (or any of the formats 
that don't require a BOM, rather than UTF-8), but be willing to accept 
any BOM-discriminated format.

* Potentially, this could be expanded beyond the various Unicode 
encodings... one could envision that a program whose data files 
historically were in any particular national language locale, could want 
to be enhance to accept Unicode, and could declare that they will accept 
any BOM-discriminated format, but want to default, in the absence of a 
BOM, to the original national language locale that they historically 
accepted.  That would provide a migration path for their old data files.

So the point is, that it might be nice to have 
"BOM-otherEncodingForDefault" for each other encoding that Python 
supports.  Not sure that is the right API, but I think it is expressive 
enough to handle the cases above.  Whether the cases solve actual 
problems or not, I couldn't say, but they seem like reasonable cases.

It would, of course, be nicest if OS metadata had been invented way back 
when, for all OSes, such that all text files were flagged with their 
encoding... then languages could just read the encoding and do the right 
thing! But we live in the real world, instead.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From python at mrabarnett.plus.com  Sat Jan  9 02:12:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 09 Jan 2010 01:12:28 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B47D7FC.4070703@mrabarnett.plus.com>

Glenn Linderman wrote:
> On approximately 1/8/2010 3:59 PM, came the following characters from 
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if 
> there is no BOM it assumes UTF-8.  That is probably a good assumption in 
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
> (cases where the endianness is unspecified) contain a BOM.  Hence, it 
> might be that someone would expect a UTF-16LE (or any of the formats 
> that don't require a BOM, rather than UTF-8), but be willing to accept 
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode 
> encodings... one could envision that a program whose data files 
> historically were in any particular national language locale, could want 
> to be enhance to accept Unicode, and could declare that they will accept 
> any BOM-discriminated format, but want to default, in the absence of a 
> BOM, to the original national language locale that they historically 
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have 
> "BOM-otherEncodingForDefault" for each other encoding that Python 
> supports.  Not sure that is the right API, but I think it is expressive 
> enough to handle the cases above.  Whether the cases solve actual 
> problems or not, I couldn't say, but they seem like reasonable cases.
> 
> It would, of course, be nicest if OS metadata had been invented way back 
> when, for all OSes, such that all text files were flagged with their 
> encoding... then languages could just read the encoding and do the right 
> thing! But we live in the real world, instead.
> 
What about listing the possible encodings? It would try each in turn
until it found one where the BOM matched or had no BOM:

     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')

or is that taking it too far?


From martin at v.loewis.de  Sat Jan  9 02:23:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 02:23:07 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47CA6F.5000607@voidspace.org.uk>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<loom.20100109T010657-648@post.gmane.org>
	<4B47CA6F.5000607@voidspace.org.uk>
Message-ID: <4B47DA7B.6050505@v.loewis.de>

>>> Antoine would like to check BOM by default, because both options
>>> (system locale vs checking for BOM) is the same thing.
>>> 
>> To be clear, I am not saying it is the same thing. What I think is 
>> that it would be a mistake to use a mildly unreliable heuristic by
>> default (the locale + device encoding heuristic) but refuse to
>> trust a more reliable heuristic (the BOM-based detection
>> algorithm).
>> 
> 
> I concur. On Windows both UTF-8 and signature are very common, yet
> the platform default is the truly awful CP1252.

While I would support combining BOM detection in the case where a file
is opened for reading and no encoding is specified, I see two problems:
a) if a seek operations is performed before having looked at the BOM,
   no determination would have been made
b) what encoding should it use on writing?

Regards,
Martin



From v+python at g.nevcal.com  Sat Jan  9 02:49:12 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 17:49:12 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <4B47E098.6030703@g.nevcal.com>

On approximately 1/8/2010 5:12 PM, came the following characters from 
the keyboard of MRAB:
> Glenn Linderman wrote:
>> On approximately 1/8/2010 3:59 PM, came the following characters from 
>> the keyboard of Victor Stinner:
>>> Hi,
>>>
>>> Thanks for all the answers! I will try to sum up all ideas here.
>>
>> One concern I have with this implementation encoding="BOM" is that if 
>> there is no BOM it assumes UTF-8.  That is probably a good assumption 
>> in some circumstances, but not in others.
>>
>> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
>> encoded files include a BOM.  It is only required that UTF-16 and 
>> UTF-32 (cases where the endianness is unspecified) contain a BOM.  
>> Hence, it might be that someone would expect a UTF-16LE (or any of 
>> the formats that don't require a BOM, rather than UTF-8), but be 
>> willing to accept any BOM-discriminated format.
>>
>> * Potentially, this could be expanded beyond the various Unicode 
>> encodings... one could envision that a program whose data files 
>> historically were in any particular national language locale, could 
>> want to be enhance to accept Unicode, and could declare that they 
>> will accept any BOM-discriminated format, but want to default, in the 
>> absence of a BOM, to the original national language locale that they 
>> historically accepted.  That would provide a migration path for their 
>> old data files.
>>
>> So the point is, that it might be nice to have 
>> "BOM-otherEncodingForDefault" for each other encoding that Python 
>> supports.  Not sure that is the right API, but I think it is 
>> expressive enough to handle the cases above.  Whether the cases solve 
>> actual problems or not, I couldn't say, but they seem like reasonable 
>> cases.
>>
>> It would, of course, be nicest if OS metadata had been invented way 
>> back when, for all OSes, such that all text files were flagged with 
>> their encoding... then languages could just read the encoding and do 
>> the right thing! But we live in the real world, instead.
>>
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
>
>     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

That sounds very flexible -- but in net effect it would only make 
illegal a subset of the BOM-containing encodings (those not listed) 
without making legal any additional encodings other than the non-BOM 
encoding.  Whether prohibiting a subset of BOM-containing encodings is a 
useful use case, I couldn't say... but my goal would be to included as 
many different file encodings on input as possible: without a BOM, that 
is exactly 1 (unless there are other heuristics), with a BOM, it is 
1+all-BOM-containing encodings.  Your scheme would permit numbers of 
encodings accepted to vary between 1 and 1+all-BOM-containing encodings.

(I think everyone can agree there are 5 different byte sequences that 
can be called a Unicode BOM.  The likelihood of them appearing in any 
other text encoding created by mankind depends on those other encodings 
-- but it is not impossible.  It is truly up to the application to 
decide whether BOM detection could potentially conflict with files in 
some other encoding that would be acceptable to the application.)

So I think it is taking it further than I can see value in, but I'm 
willing to be convinced otherwise.  I see only a need for detecting BOM, 
and specifying a default encoding to be used if there is no BOM.  Note 
that it might be nice to have a specification for using current 
encoding=None heuristic -- perhaps encoding="BOM-None" per my originally 
proposed syntax.  But I'm still not saying that is the best syntax.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From ncoghlan at gmail.com  Sat Jan  9 04:07:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Jan 2010 13:07:12 +1000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B476196.4080409@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>
	<4B476196.4080409@mrabarnett.plus.com>
Message-ID: <4B47F2E0.7090403@gmail.com>

MRAB wrote:
> Maybe there should also be a way of determining what encoding it decided
> it was, so that you can then write a new file in that same encoding.

I thought of that question as well - the f.encoding attribute on the
opened file should be sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Sat Jan  9 06:48:36 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 06:48:36 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47E098.6030703@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com> <4B47D7FC.4070703@mrabarnett.plus.com>
	<4B47E098.6030703@g.nevcal.com>
Message-ID: <319e029f1001082148h5cee2c0bn76f70a17766ca69e@mail.gmail.com>

It seems to me that when opening a file, the following is the only
flow that makes sense for the typical opening of a file flow:

if encoding is not None:
   use encoding
elif file has BOM:
   use BOM
else:
   use system default

And hence a encoding='BOM' isn't needed there. Although I'm trying to
come up with usecases that doesn't work with this, I can't. :)

BUT

When writing things are not so easy though. Apparently some encodings
require a BOM to be written, but others do not, but allow it, and some
has no byte order mark. So there you have to be able to write the BOM,
or not. And that's either a new parameter, because you can't use
encoding='BOM' since you need to specify the encoding as well, or a
new method.

I would suggest a BOM parameter, and maybe a method as  well.

BOM=None|True|False

Where "None" means a sane default behaviour, that is write a BOM if
the encoding require it.
"True" means write a BOM if the encoding *supports* it.
"False" means Don't write a BOM even if the encoding requires it
(because I know what I'm doing)

if 'w' in mode: # But not 'r' or 'a'
    if BOM == True and encoding in (ENCODINGS THAT ALLOW BOM):
        write_bom = True
    elif BOM == False:
       write_bom = False
    elif BOM == None and encoding in (ENCODINGS THAT REQUIRE BOM):
          write_bom = True
    else:
          write_bom = False
else:
    write_bom = False

For reading this parameter could either be a noop, or possibly change
the behavior somehow, if a usecase where that makes sense can be
imagined.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From walter at livinglogic.de  Sat Jan  9 11:51:57 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 11:51:57 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B485FCD.2040901@livinglogic.de>

On 09.01.10 01:47, Glenn Linderman wrote:

> On approximately 1/8/2010 3:59 PM, came the following characters from
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.  That is probably a good assumption in
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32
> (cases where the endianness is unspecified) contain a BOM.  Hence, it
> might be that someone would expect a UTF-16LE (or any of the formats
> that don't require a BOM, rather than UTF-8), but be willing to accept
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode
> encodings... one could envision that a program whose data files
> historically were in any particular national language locale, could want
> to be enhance to accept Unicode, and could declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have
> "BOM-otherEncodingForDefault" for each other encoding that Python
> supports.  Not sure that is the right API, but I think it is expressive
> enough to handle the cases above.  Whether the cases solve actual
> problems or not, I couldn't say, but they seem like reasonable cases.

This is doable with the currect API. Simply define a codec search
function that handles all encoding names that start with "BOM-" and pass
the "otherEncodingForDefault" part along to the codec.

> It would, of course, be nicest if OS metadata had been invented way back
> when, for all OSes, such that all text files were flagged with their
> encoding... then languages could just read the encoding and do the right
> thing! But we live in the real world, instead.

Servus,
   Walter


From walter at livinglogic.de  Sat Jan  9 12:18:33 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 12:18:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081140.28124.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
Message-ID: <4B486609.2050804@livinglogic.de>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
>>> Builtin open() function is unable to open an UTF-16/32 file starting with
>>> a BOM if the encoding is not specified (raise an unicode error). For an
>>> UTF-8 file starting with a BOM, read()/readline() returns also the BOM
>>> whereas the BOM should be "ignored".
>> It depends. If you use the utf-8-sig encoding, it *will* ignore the
>> UTF-8 signature.
> 
> Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
> UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
> remove the BOM after the first read (much harder if you use a module like 
> ConfigParser or csv).
> 
>>> Since my proposition changes the result TextIOWrapper.read()/readline()
>>> for files starting with a BOM, we might introduce an option to open() to
>>> enable the new behaviour. But is it really needed to keep the backward
>>> compatibility?
>> Absolutely. And there is no need to produce a new option, but instead
>> use the existing options: define an encoding that auto-detects the
>> encoding from the family of BOMs. Maybe you call it encoding="sniff".
> 
> Good idea, I choosed open(filename, encoding="BOM").

On the surface this looks like there's an encoding named "BOM", but 
looking at your patch I found that the check is still done in 
TextIOWrapper. IMHO the best approach would to the implement a *real* 
codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
the IO library. It could even be developed as a standalone project and 
published in the Cheeseshop.

To see how something like this can be done, take a look at the UTF-16 
codec, that switches to bigendian or littleendian mode depending on the 
first read/decode call.

Servus,
    Walter







From victor.stinner at haypocalc.com  Sat Jan  9 13:37:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:37:06 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47DA7B.6050505@v.loewis.de>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47CA6F.5000607@voidspace.org.uk> <4B47DA7B.6050505@v.loewis.de>
Message-ID: <201001091337.06596.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:23:07, Martin v. L?wis a ?crit :
> While I would support combining BOM detection in the case where a file
> is opened for reading and no encoding is specified, I see two problems:
> a) if a seek operations is performed before having looked at the BOM,
>    no determination would have been made

TextIOWrapper doesn't support seek to an arbitrary byte. It uses "cookie" 
which is an opaque value. Reuse a cookie from another file or an old cookie is 
forbidden (but it doesn't raise an error). This is not specific to the BOM 
checking: the problem already exist for encodings using a BOM (eg. UTF-16).

> b) what encoding should it use on writing?

Don't change anything to writing.

With Antoince choice: open('file.txt', 'w', encoding=None) continue to use the 
actual heuristic (os.device_encoding() or system locale).

With Guido choice, encoding="BOM": it raises an error, because BOM check is 
not supported when writing into a file. How could the BOM be checked when 
creating a new (empty) file!?

-- 
Victor Stinner
http://www.haypocalc.com/


From mal at egenix.com  Sat Jan  9 13:45:58 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 09 Jan 2010 13:45:58 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B487A86.4020603@egenix.com>

Victor Stinner wrote:
> (2) Check for a BOM while reading or detect it before?
> 
> Everybody agree that checking BOM is an interesting option and should not be 
> limited to open().
> 
> Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
> name or a binary file-like object: it returns the encoding and seek to the 
> file start or just after the BOM.
> 
> I dislike this function because it requires extra file operations (open 
> (optional), read() and seek()) and it doesn't work if the file is not seekable 
> (eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
> avoid extra file operations.
> 
> Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
> any file-like object.

Yes, but the implementation is limited to just BOM checking
and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

With a codecs module function we could easily extend the
encoding detection to more file types, e.g. XML files,
Python source code files, etc. that use other mechanisms
for defining the encoding.

BTW: I haven't looked at your implementation, but what happens
when your BOM check fails ? Will the implementation add the
already read bytes back to a buffer ?

This rollback action is the only reason for needing a
seekable stream in codecs.guess_stream_encoding().

Another point to consider:

AFAIK, we currently have a moratorium on changes to Python
builtins. How does that match up with the proposed changes ?

Using a new codec like Walter suggested would move the
implementation into the stdlib for which doesn't the
moratorium doesn't apply.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From victor.stinner at haypocalc.com  Sat Jan  9 13:54:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:54:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <201001091354.17239.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 01:47:38, vous avez ?crit :
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.

If no BOM is found, it fallback to the current heuristic: os.device_encoding() 
or system local.

> (...) Hence, it might be that someone would expect a UTF-16LE (or any of 
> the formats that don't require a BOM, rather than UTF-8), but be willing 
> to accept any BOM-discriminated format.
> (...) declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted

You mean "if there is a BOM, use it, otherwise fallback to a specific 
charset"? How could it be declared? Maybe:

   open("file.txt", check_bom=True, encoding="UTF16-LE")
   open("file.txt", check_bom=True, encoding="latin1")

About falling back to UTF-8, it would be written:

   open("file.txt", check_bom=True, encoding="UTF-8")

As explained before, check_bom=True is only accepted for read only file mode.

Well, why not. This is a third choice for my point (1) :-) It's between Guido 
and Antoine choice, and I like it because we can fallback to UTF-8 instead of 
the dummy system locale: Windows users will be happy to be able to use UTF-8 
:-) I prefer to fallback to a fixed encoding then depending on the system 
locale.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:34:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:34:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <201001091434.17521.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:12:28, MRAB a ?crit :
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
> 
>      my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

Yes, you're taking it foo far :-) Checking BOM is reliable, whereas *guessing* 
the charset only using the byte stream can only be an heuristic. Guess a 
charset is a complex problem, they are 3rd party library to do that, like the 
chardet project.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:38:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:38:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B486609.2050804@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <201001091438.43576.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
> > Good idea, I choosed open(filename, encoding="BOM").
> 
> On the surface this looks like there's an encoding named "BOM", but
> looking at your patch I found that the check is still done in
> TextIOWrapper. IMHO the best approach would to the implement a *real*
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
> the IO library. It could even be developed as a standalone project and
> published in the Cheeseshop.

Why not, this is another solution to the point (2) (Check for a BOM while 
reading or detect it before?). Which encoding would be used if there is not 
BOM? UTF-8 sounds like a good choice.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:50:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:50:28 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B487A86.4020603@egenix.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B487A86.4020603@egenix.com>
Message-ID: <201001091450.28497.victor.stinner@haypocalc.com>

Hi,

Le samedi 09 janvier 2010 13:45:58, vous avez ?crit :
> > Note: I implemented the BOM check in TextIOWrapper; so it's already
> > usable for any file-like object.
> 
> Yes, but the implementation is limited to just BOM checking
> and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

Sure, but that's already better than no BOM check :-) It looks like many 
people would apprecite UTF-8-SIG detection, since this encoding is common on 
Windows.

> BTW: I haven't looked at your implementation, but what happens
> when your BOM check fails ? Will the implementation add the
> already read bytes back to a buffer ?

My implementation is done between buffer.read() and decoder.decode(data). If 
there is a BOM: set the encoding and remove the BOM bytes from the byte 
string. Otherwise, use another algorithm to choose the encoding and leave the 
byte string unchanged.

It can be seen as a codec: it works like UTF-16 and UTF-32 codecs ;-)

> AFAIK, we currently have a moratorium on changes to Python
> builtins. How does that match up with the proposed changes ?

Oh yes, I forgot the moratorium. In all solutions, some of them don't change 
the API. Eg. Antoine proposed to leave the API unchanged: open(file) => 
open(file) :-) I don't know if it's compatible with the moratorium or not.

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 16:05:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 15:05:45 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <loom.20100109T160248-501@post.gmane.org>

Walter D?rwald <walter <at> livinglogic.de> writes:
> 
> On the surface this looks like there's an encoding named "BOM", but 
> looking at your patch I found that the check is still done in 
> TextIOWrapper. IMHO the best approach would to the implement a *real* 
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
> the IO library. It could even be developed as a standalone project and 
> published in the Cheeseshop.

Sorry but this is missing the point. The point here is to improve the open()
function. I'm sure people who know about encodings are able to install the
chardet library or even whip up their own BOM detection routine...


Antoine.




From benjamin at python.org  Sat Jan  9 18:29:33 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 11:29:33 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>

On behalf of the Python development team, I'm gleeful to announce the second
alpha release of Python 2.7.

Python 2.7 is scheduled to be the last major version in the 2.x series.  It
includes many features that were first released in Python 3.1.  The faster io
module, the new nested with statement syntax, improved float repr, and the
memoryview object have been backported from 3.1. Other features include an
ordered dictionary implementation, unittests improvements, and support for ttk
Tile in Tkinter.  For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7 visit:

     http://www.python.org/download/releases/2.7/

Please note that this is a development release, intended as a preview of new
features for the community, and is thus not suitable for production use.

The 2.7 documentation can be found at:

     http://docs.python.org/2.7

Please consider trying Python 2.7 with your code and reporting any bugs you may
notice to:

     http://bugs.python.org


Have fun!

--
Benjamin Peterson
2.7 Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7's contributors)


From kmtracey at gmail.com  Sat Jan  9 19:48:12 2010
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 9 Jan 2010 13:48:12 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>

On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On behalf of the Python development team, I'm gleeful to announce the
> second
> alpha release of Python 2.7.
>
>
Well yay.  Django's test suite (1242 tests) runs with just one failure on
the 2.7 alpha 2 level, and that looks to be likely due to the improved
string/float rounding so not really a problem, just a difference.  That's
down from 104 failures and 40 errors with 2.7 alpha 1.

Note on the website page http://www.python.org/download/releases/2.7/ the
"Change log for this release" link is still pointing to the alpha 1
changelog.

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100109/d5c06f60/attachment-0005.htm>

From benjamin at python.org  Sat Jan  9 19:51:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 12:51:11 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <1afaf6161001091051kb1ba08q9b96094af16c12fa@mail.gmail.com>

2010/1/9 Karen Tracey <kmtracey at gmail.com>:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>
> Well yay.? Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.? That's
> down from 104 failures and 40 errors with 2.7 alpha 1.

Excellent!

>
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Thanks. I'll fix that.

>



-- 
Regards,
Benjamin


From skip at pobox.com  Sat Jan  9 21:00:44 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:00:44 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
Message-ID: <19272.57452.972234.643008@montanaro.dyndns.org>

How much of the Unladen Swallow cPickle speedups have been incorporated into
2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
two versions I currently care about at work - we will skip 2.5 entirely).
It appears some of their speedups may have already been merged to trunk, but
I'm not sure how much.  If a patch to merge this to 2.7 is already under
consideration I won't look at it, but I interpreted Collin Winter's response
to my query on the u-s mailing list that not everything has been done yet.

Thx,

Skip



From martin at v.loewis.de  Sat Jan  9 21:09:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 09 Jan 2010 21:09:27 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T160248-501@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
Message-ID: <4B48E277.9010408@v.loewis.de>

Antoine Pitrou wrote:
> Walter D?rwald <walter <at> livinglogic.de> writes:
>> On the surface this looks like there's an encoding named "BOM", but 
>> looking at your patch I found that the check is still done in 
>> TextIOWrapper. IMHO the best approach would to the implement a *real* 
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
>> the IO library. It could even be developed as a standalone project and 
>> published in the Cheeseshop.
> 
> Sorry but this is missing the point. The point here is to improve the open()
> function. I'm sure people who know about encodings are able to install the
> chardet library or even whip up their own BOM detection routine...

How does the requirement that it be implemented as a codec miss the
point?

FWIW, I agree with Walter that if it is provided through the encoding=
argument, it should be a codec. If it is built into the open function
(for whatever reason), it must be provided by some other parameter.

I do see the point that it becomes available to end users only when
released as part of Python. However, this *also* means that applications
won't be using it for another three years or so, since they'll have to
support older Python versions as well (unless it is integrated in the
case where no encoding is specified).

Regards,
Martin


From pjenvey at underboss.org  Sat Jan  9 21:09:29 2010
From: pjenvey at underboss.org (Philip Jenvey)
Date: Sat, 9 Jan 2010 12:09:29 -0800
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <19272.57452.972234.643008@montanaro.dyndns.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>


On Jan 9, 2010, at 12:00 PM, skip at pobox.com wrote:

> How much of the Unladen Swallow cPickle speedups have been incorporated into
> 2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
> two versions I currently care about at work - we will skip 2.5 entirely).
> It appears some of their speedups may have already been merged to trunk, but
> I'm not sure how much.  If a patch to merge this to 2.7 is already under
> consideration I won't look at it, but I interpreted Collin Winter's response
> to my query on the u-s mailing list that not everything has been done yet.

They've documented their upstream patches here:

http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

--
Philip Jenvey



From skip at pobox.com  Sat Jan  9 21:21:20 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:21:20 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
Message-ID: <19272.58688.173011.412712@montanaro.dyndns.org>


    Philip> They've documented their upstream patches here:

    Philip> http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

Thanks.  That will help immensely.

Skip



From solipsis at pitrou.net  Sat Jan  9 21:28:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:28:12 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
Message-ID: <loom.20100109T212555-508@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.




From solipsis at pitrou.net  Sat Jan  9 21:32:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:32:27 +0000 (UTC)
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <loom.20100109T213033-976@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> If a patch to merge this to 2.7 is already under
> consideration I won't look at it,

Why won't you look at it? :)
Actually, if these patches are to be merged someone should certainly look at
them, and do the (possibly) remaining work.

http://bugs.python.org/issue5683
http://bugs.python.org/issue5671

Thank you

Antoine.




From skip at pobox.com  Sat Jan  9 21:43:42 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:43:42 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
In-Reply-To: <loom.20100109T213033-976@post.gmane.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<loom.20100109T213033-976@post.gmane.org>
Message-ID: <19272.60030.25319.269597@montanaro.dyndns.org>

>>>>> "Antoine" == Antoine Pitrou <solipsis at pitrou.net> writes:

    Antoine> <skip <at> pobox.com> writes:
    >> 
    >> If a patch to merge this to 2.7 is already under
    >> consideration I won't look at it,

    Antoine> Why won't you look at it? :)

I meant I wouldn't look at developing one.

Skip


From regebro at gmail.com  Sat Jan  9 23:14:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 23:14:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>

On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Right. It seems like encoding=None is the right way to go there.
encoding='BOM' would probably only work if 'BOM' isn't an encoding but
a special tag, which is ugly.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Sun Jan 10 00:25:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 23:25:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
Message-ID: <4B49105E.303@voidspace.org.uk>

On 09/01/2010 22:14, Lennart Regebro wrote:
> On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>    
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
>>      
> Right. It seems like encoding=None is the right way to go there.
> encoding='BOM' would probably only work if 'BOM' isn't an encoding but
> a special tag, which is ugly.
>
>    
I would rather see it as the default behavior for open without an 
encoding specified.

I know Guido has expressed a preference against this so I won't continue 
to flog it.

The current behavior however is that we have a 'guessing' algorithm 
based on the platform default. Currently if you open a text file in read 
mode that has a UTF-8 signature, but the platform default is something 
other than UTF-8, then we open the file using what is likely to be the 
incorrect encoding. Looking for the signature seems to be better 
behaviour in that case.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Sun Jan 10 00:40:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Jan 2010 00:40:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <4B4913DF.7050801@v.loewis.de>

>> How does the requirement that it be implemented as a codec miss the
>> point?
> 
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Yes - however, Victor currently apparently *doesn't* want it to be the
default, but wants the user to specify encoding="BOM". If so, it isn't
the default, and it is easy to implement as a codec.

>> FWIW, I agree with Walter that if it is provided through the encoding=
>> argument, it should be a codec. If it is built into the open function
>> (for whatever reason), it must be provided by some other parameter.
> 
> Why not simply encoding=None?

I don't mind. Please re-read Walter's message - it only said that
*if* this is activated through encoding="BOM", *then* it must be
a codec, and could be on PyPI. I don't think Walter was talking about
the case "it is not activated through encoding='BOM'" *at all*.

> The default value should provide the most useful
> behaviour possible. Forcing users to choose between two different autodetection
> strategies (encoding=None and another one) is a little insane IMO.

That wouldn't disturb me much. There are a lot of things in that area
that are a little insane, starting with Microsoft Windows :-)

Regards,
Martin


From henning.vonbargen at arcor.de  Sun Jan 10 12:10:02 2010
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 10 Jan 2010 12:10:02 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <mailman.2987.1263079529.28904.python-dev@python.org>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
Message-ID: <4B49B58A.4040103@arcor.de>

If Python should support BOM when reading text files,
it should also be able to *write* such files.

An encoding="BOM" argument wouldn't help here, because
it does not specify which encoding to use actually:
UFT-8, UTF-16-LE or what?

That would be a point against encoding="BOM" and
pro an additional keyword argument "use_bom" or whatever
with the following values:

None: default (old) behaviour: don't handle BOM at all

True: reading: expect BOM (raising an exception if it's
                missing). The encoding argument must be None
                or it must match the encoding implied by the
                BOM
       writing: write a BOM. The encoding argument must be
                one of the UTF encodings.
False: reading: If a BOM is present, use it to determine the
                file encoding. The encoding argument must
                be None or it must match the encoding implied by
                the BOM. (*)
                Otherwise, use the encoding argument to determine
                the encoding.
        writing: do not write a BOM. Use the encoding argument.

(*) This is a question of taste. I think some people would prefer
     a fourth value "AUTO" instead, or to swap the behaviour of
     None and False.

Henning

P.S. To make things worse, I have sometimes seen XML files with a
UTF-8 BOM, but an XML encoding declaration of "iso-8859-1".
For such files, whatever you guess will be wrong anyway...


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From brett at python.org  Sun Jan 10 19:51:26 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 10:51:26 -0800
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
Message-ID: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>

Nick Coghlan thought I should forward this to python-dev so people are aware
of this change and why it occurred (plus it indirectly informs Guido I
finally finished the work).

---------- Forwarded message ----------
From: brett.cannon <python-checkins at python.org>
Date: Sat, Jan 9, 2010 at 18:56
Subject: [Python-checkins] r77402 - in python/trunk:
Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS Python/_warnings.c
To: python-checkins at python.org


Author: brett.cannon
Date: Sun Jan 10 03:56:19 2010
New Revision: 77402

Log:
DeprecationWarning is now silent by default.

This was originally suggested by Guido, discussed on the stdlib-sig mailing
list, and given the OK by Guido directly to me. What this change essentially
means is that Python has taken a policy of silencing warnings that are only
of interest to developers by default. This should prevent users from seeing
warnings which are triggered by an application being run against a new
interpreter before the app developer has a chance to update their code.

Closes issue #7319. Thanks to Antoine Pitrou, Ezio Melotti, and Brian Curtin
for helping with the issue.


Modified:
  python/trunk/Doc/library/warnings.rst
  python/trunk/Lib/test/test_ascii_formatd.py
  python/trunk/Lib/test/test_exceptions.py
  python/trunk/Lib/warnings.py
  python/trunk/Misc/NEWS
  python/trunk/Python/_warnings.c

Modified: python/trunk/Doc/library/warnings.rst
==============================================================================
--- python/trunk/Doc/library/warnings.rst       (original)
+++ python/trunk/Doc/library/warnings.rst       Sun Jan 10 03:56:19 2010
@@ -59,7 +59,7 @@
 | :exc:`UserWarning`               | The default category for :func:`warn`.
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`DeprecationWarning`        | Base category for warnings about
deprecated   |
-|                                  | features.
        |
+|                                  | features (ignored by default).
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`SyntaxWarning`             | Base category for warnings about
dubious      |
 |                                  | syntactic features.
        |
@@ -89,6 +89,9 @@
 standard warning categories.  A warning category must always be a subclass
of
 the :exc:`Warning` class.

+.. versionchanged:: 2.7
+   :exc:`DeprecationWarning` is ignored by default.
+

 .. _warning-filter:

@@ -148,14 +151,6 @@
 :mod:`warnings` module parses these when it is first imported (invalid
options
 are ignored, after printing a message to ``sys.stderr``).

-The warnings that are ignored by default may be enabled by passing
:option:`-Wd`
-to the interpreter. This enables default handling for all warnings,
including
-those that are normally ignored by default. This is particular useful for
-enabling ImportWarning when debugging problems importing a developed
package.
-ImportWarning can also be enabled explicitly in Python code using::
-
-   warnings.simplefilter('default', ImportWarning)
-

 .. _warning-suppress:

@@ -226,6 +221,37 @@
 entries from the warnings list before each new operation).


+Updating Code For New Versions of Python
+----------------------------------------
+
+Warnings that are only of interest to the developer are ignored by default.
As
+such you should make sure to test your code with typically ignored warnings
+made visible. You can do this from the command-line by passing
:option:`-Wd`
+to the interpreter (this is shorthand for :option:`-W default`).  This
enables
+default handling for all warnings, including those that are ignored by
default.
+To change what action is taken for encountered warnings you simply change
what
+argument is passed to :option:`-W`, e.g. :option:`-W error`. See the
+:option:`-W` flag for more details on what is possible.
+
+To programmatically do the same as :option:`-Wd`, use::
+
+  warnings.simplefilter('default')
+
+Make sure to execute this code as soon as possible. This prevents the
+registering of what warnings have been raised from unexpectedly influencing
how
+future warnings are treated.
+
+Having certain warnings ignored by default is done to prevent a user from
+seeing warnings that are only of interest to the developer. As you do not
+necessarily have control over what interpreter a user uses to run their
code,
+it is possible that a new version of Python will be released between your
+release cycles.  The new interpreter release could trigger new warnings in
your
+code that were not there in an older interpreter, e.g.
+:exc:`DeprecationWarning` for a module that you are using. While you as a
+developer want to be notified that your code is using a deprecated module,
to a
+user this information is essentially noise and provides no benefit to them.
+
+
 .. _warning-functions:

 Available Functions

Modified: python/trunk/Lib/test/test_ascii_formatd.py
==============================================================================
--- python/trunk/Lib/test/test_ascii_formatd.py (original)
+++ python/trunk/Lib/test/test_ascii_formatd.py Sun Jan 10 03:56:19 2010
@@ -4,6 +4,7 @@

 import unittest
 from test_support import check_warnings, run_unittest, cpython_only
+import warnings


 class FormatDeprecationTests(unittest.TestCase):
@@ -17,6 +18,7 @@
        buf = create_string_buffer(' ' * 100)

        with check_warnings() as w:
+            warnings.simplefilter('default')
            PyOS_ascii_formatd(byref(buf), sizeof(buf), '%+.10f',
                               c_double(10.0))
            self.assertEqual(buf.value, '+10.0000000000')

Modified: python/trunk/Lib/test/test_exceptions.py
==============================================================================
--- python/trunk/Lib/test/test_exceptions.py    (original)
+++ python/trunk/Lib/test/test_exceptions.py    Sun Jan 10 03:56:19 2010
@@ -309,6 +309,7 @@
        # BaseException.__init__ triggers a deprecation warning.
        exc = BaseException("foo")
        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('default')
            self.assertEquals(exc.message, "foo")
        self.assertEquals(len(w), 1)
        self.assertEquals(w[0].category, DeprecationWarning)

Modified: python/trunk/Lib/warnings.py
==============================================================================
--- python/trunk/Lib/warnings.py        (original)
+++ python/trunk/Lib/warnings.py        Sun Jan 10 03:56:19 2010
@@ -383,8 +383,8 @@
 # Module initialization
 _processoptions(sys.warnoptions)
 if not _warnings_defaults:
-    simplefilter("ignore", category=PendingDeprecationWarning, append=1)
-    simplefilter("ignore", category=ImportWarning, append=1)
+    for cls in (DeprecationWarning, PendingDeprecationWarning,
ImportWarning):
+        simplefilter("ignore", category=cls, append=True)
    bytes_warning = sys.flags.bytes_warning
    if bytes_warning > 1:
        bytes_action = "error"

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sun Jan 10 03:56:19 2010
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------

+- Issue #7319: Silence DeprecationWarning by default.
+
 - Issue #2335: Backport set literals syntax from Python 3.x.

 Library

Modified: python/trunk/Python/_warnings.c
==============================================================================
--- python/trunk/Python/_warnings.c     (original)
+++ python/trunk/Python/_warnings.c     Sun Jan 10 03:56:19 2010
@@ -85,10 +85,10 @@

    default_action = get_warnings_attr("defaultaction");
    if (default_action == NULL) {
-       if (PyErr_Occurred()) {
-           return NULL;
-       }
-       return _default_action;
+        if (PyErr_Occurred()) {
+            return NULL;
+        }
+        return _default_action;
    }

    Py_DECREF(_default_action);
@@ -202,12 +202,12 @@

    mod_str = PyString_AsString(filename);
    if (mod_str == NULL)
-           return NULL;
+        return NULL;
    len = PyString_Size(filename);
    if (len < 0)
        return NULL;
    if (len >= 3 &&
-       strncmp(mod_str + (len - 3), ".py", 3) == 0) {
+            strncmp(mod_str + (len - 3), ".py", 3) == 0) {
        module = PyString_FromStringAndSize(mod_str, len-3);
    }
    else {
@@ -251,7 +251,7 @@

    name = PyObject_GetAttrString(category, "__name__");
    if (name == NULL)  /* XXX Can an object lack a '__name__' attribute? */
-           return;
+        return;

    f_stderr = PySys_GetObject("stderr");
    if (f_stderr == NULL) {
@@ -341,7 +341,7 @@
        rc = already_warned(registry, key, 0);
        if (rc == -1)
            goto cleanup;
-       else if (rc == 1)
+        else if (rc == 1)
            goto return_none;
        /* Else this warning hasn't been generated before. */
    }
@@ -492,9 +492,9 @@
    /* Setup filename. */
    *filename = PyDict_GetItemString(globals, "__file__");
    if (*filename != NULL) {
-           Py_ssize_t len = PyString_Size(*filename);
+            Py_ssize_t len = PyString_Size(*filename);
        const char *file_str = PyString_AsString(*filename);
-           if (file_str == NULL || (len < 0 && PyErr_Occurred()))
+            if (file_str == NULL || (len < 0 && PyErr_Occurred()))
            goto handle_error;

        /* if filename.lower().endswith((".pyc", ".pyo")): */
@@ -506,10 +506,10 @@
                tolower(file_str[len-1]) == 'o'))
        {
            *filename = PyString_FromStringAndSize(file_str, len-1);
-               if (*filename == NULL)
-                       goto handle_error;
-           }
-           else
+            if (*filename == NULL)
+                goto handle_error;
+        }
+        else
            Py_INCREF(*filename);
    }
    else {
@@ -536,8 +536,8 @@
            else {
                /* embedded interpreters don't have sys.argv, see bug
#839151 */
                *filename = PyString_FromString("__main__");
-                   if (*filename == NULL)
-                       goto handle_error;
+                if (*filename == NULL)
+                    goto handle_error;
            }
        }
        if (*filename == NULL) {
@@ -839,26 +839,29 @@
 static PyObject *
 init_filters(void)
 {
-    PyObject *filters = PyList_New(3);
+    PyObject *filters = PyList_New(4);
    const char *bytes_action;
    if (filters == NULL)
        return NULL;

    PyList_SET_ITEM(filters, 0,
+                    create_filter(PyExc_DeprecationWarning, "ignore"));
+    PyList_SET_ITEM(filters, 1,
                    create_filter(PyExc_PendingDeprecationWarning,
"ignore"));
-    PyList_SET_ITEM(filters, 1, create_filter(PyExc_ImportWarning,
"ignore"));
+    PyList_SET_ITEM(filters, 2, create_filter(PyExc_ImportWarning,
"ignore"));
    if (Py_BytesWarningFlag > 1)
        bytes_action = "error";
    else if (Py_BytesWarningFlag)
        bytes_action = "default";
    else
        bytes_action = "ignore";
-    PyList_SET_ITEM(filters, 2, create_filter(PyExc_BytesWarning,
+    PyList_SET_ITEM(filters, 3, create_filter(PyExc_BytesWarning,
                    bytes_action));

    if (PyList_GET_ITEM(filters, 0) == NULL ||
        PyList_GET_ITEM(filters, 1) == NULL ||
-        PyList_GET_ITEM(filters, 2) == NULL) {
+        PyList_GET_ITEM(filters, 2) == NULL ||
+        PyList_GET_ITEM(filters, 3) == NULL) {
        Py_DECREF(filters);
        return NULL;
     }
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db6da5fd/attachment-0005.htm>

From victor.stinner at haypocalc.com  Sun Jan 10 20:22:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 10 Jan 2010 20:22:10 +0100
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
Message-ID: <201001102022.10259.victor.stinner@haypocalc.com>

Hi,

Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
> Nick Coghlan thought I should forward this to python-dev so people are
>  aware of this change and why it occurred (plus it indirectly informs Guido
>  I finally finished the work).

Thanks to copy this mail to the python-dev mailing list.

What is the recommanded way to get the previous behaviour (print deprecation 
warnings once)? Is there a command line option? Is it "-W default"?

-- 
Victor Stinner
http://www.haypocalc.com/


From benjamin at python.org  Sun Jan 10 20:23:54 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 13:23:54 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <201001102022.10259.victor.stinner@haypocalc.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
	<201001102022.10259.victor.stinner@haypocalc.com>
Message-ID: <1afaf6161001101123g366d80dbjeaa80f1a632605e2@mail.gmail.com>

2010/1/10 Victor Stinner <victor.stinner at haypocalc.com>:
> Hi,
>
> Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
>> Nick Coghlan thought I should forward this to python-dev so people are
>> ?aware of this change and why it occurred (plus it indirectly informs Guido
>> ?I finally finished the work).
>
> Thanks to copy this mail to the python-dev mailing list.
>
> What is the recommanded way to get the previous behaviour (print deprecation
> warnings once)? Is there a command line option? Is it "-W default"?

That commit conveniently adds documentation answering that question. :)



-- 
Regards,
Benjamin


From nas at arctrix.com  Sun Jan 10 20:30:09 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:30:09 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <hid9s1$i05$1@ger.gmane.org>

Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm gleeful to announce
> the second alpha release of Python 2.7.

Thanks to everyone who contributed.

> Python 2.7 is scheduled to be the last major version in the 2.x
> series.

Has this really been decided already? Maybe I missed it.  In my
opinion, it does Python's reputation harm to make such a statement.
Conservative users (probably the vast majority of Python users)
don't like to hear that software they are considering using is
nearing the end of its life.  What does it gain us to announce that
the 2.x branch is dead aside from bugfixes?

I propose that the 2.x branch be treated like 2.x.y branches: as
long as there is sufficient volunteer labour, it should continue to
live.  In order to avoid wasted development effort, it would be
prudent to announce that unless a 2.8 release manager steps up,
whatever is committed to the 2.x branch after the 2.7 release may
never get released.

Said another way, it's okay for the Python developers to decide to
abandon 2.x and put their efforts into 3.x. It's not okay for them
to prevent others from continuing to work on 2.x or to somehow make
2.x look worse so 3.x looks better. Python 3 needs to stand on its
own terms and I'm confident it can.

Best regards,

  Neil



From brett at python.org  Sun Jan 10 21:09:08 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:09:08 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>

On Sun, Jan 10, 2010 at 11:30, Neil Schemenauer <nas at arctrix.com> wrote:

> Benjamin Peterson <benjamin at python.org> wrote:
> > On behalf of the Python development team, I'm gleeful to announce
> > the second alpha release of Python 2.7.
>
> Thanks to everyone who contributed.
>
> > Python 2.7 is scheduled to be the last major version in the 2.x
> > series.
>
> Has this really been decided already? Maybe I missed it.


More or less. It was first discussed at the language summit last year and
has come up here a couple of times. If needed we can make it official in
terms of lifetime of 2.7, etc. at the language summit this year.


>  In my
> opinion, it does Python's reputation harm to make such a statement.
> Conservative users (probably the vast majority of Python users)
> don't like to hear that software they are considering using is
> nearing the end of its life.  What does it gain us to announce that
> the 2.x branch is dead aside from bugfixes?
>
> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.
>
> Said another way, it's okay for the Python developers to decide to
> abandon 2.x and put their efforts into 3.x. It's not okay for them
> to prevent others from continuing to work on 2.x or to somehow make
> 2.x look worse so 3.x looks better. Python 3 needs to stand on its
> own terms and I'm confident it can.
>

I don't think ending the 2.x series at 2.7 makes it look bad compared to
3.2; it's simply the end of a development line like any other software
project. I suspect 2.7 will have a protracted bugfix window because so much
code runs on 2.x exclusively at the moment. And if core developers want to
continue to backport fixes past two years  from release they can (or however
long we decide to officially support 2.7).

No one is saying people still can't work on the code, just that python-dev
as an entity is not going to focus its effort on the 2.x series anymore and
people should not rely upon us to continue to focus new development efforts
in that series. If there are core developers who want to continue to do
bugfix releases then that's fine and I am happy to flag patches as needing
backports and let other's do the work after the standard two year
maintenance cycle, but I know I do not want to be held accountable as a core
developer to keep the 2.x going indefinitely. Maintaining four branches is
enough of a reason in my book to not keep the 2.x series going.

If there really is an outcry on this we can re-visit the issue, but as of
right now we need to move forward at some point and 2.7 seems like that good
point.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/4bff0434/attachment-0005.htm>

From ncoghlan at gmail.com  Sun Jan 10 21:23:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 11 Jan 2010 06:23:27 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A373F.9050601@gmail.com>

Neil Schemenauer wrote:
> In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

The announcement is precisely to avoid the situation where people commit
new features to the 2.x main line of development (after the 2.7
maintenance branch is created) in the expectation that they will be
released as part of a hypothetical 2.8 release.

Whether that info needs to be in each and every 2.7 announcement...
probably not. It isn't really info for users of Python, just for
developers of Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Sun Jan 10 21:25:29 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:25:29 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
Message-ID: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>

Michael has given me the hg transition/stdlib time slot at the language
summit this year. In regards to that I plan to lead a discussion on:

* where we are at w/ the Hg transition (Dirkjan should be there and I did a
blog post on this topic recently:
http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
* argparse (PEP 389)
* brief mention on still wanting to break out the stdlib from CPython
* an official policy on extension modules? (i.e. must have a pure Python
implementation which can mean a ctypes implementation unless given an
explicit waiver)

That's everything from a stdlib perspective. I have half-baked ideas, but
nothing concrete enough to bring up unless people really want to discuss
stuff like how to potentially standardize module deprecation warnings, etc.

In terms of me personally, I do plan to bring up at some point during the
summit these points which don't squarely fit during my time slot:

* an official unofficial policy on how new proposed features should come to
us (i.e. working code to python-ideas with a shell of a PEP that includes
abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
* any changes needed to the issue tracker to help with the workflow? (stage
field seems like a failed experiment and we now have several effective
triage people who can help w/ guiding changes)

If there is something missing from the stdlib discussion that you think
should be brought up at the summit let me know. And if there is something
here you want to discuss before the summit let me know and I can start a
separate thread on it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/f16584e2/attachment-0005.htm>

From dirkjan at ochtman.nl  Sun Jan 10 22:52:00 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 10 Jan 2010 22:52:00 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:25, Brett Cannon <brett at python.org> wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)

Unfortunately my flight doesn't land until about the time the
Mercurial slot ends, so while I'll be there later on that afternoon
for sprinting or questions or anything, I won't be at the actual
Mercurial talk at the summit.

Cheers,

Dirkjan


From fuzzyman at voidspace.org.uk  Sun Jan 10 23:27:58 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 10 Jan 2010 22:27:58 +0000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
Message-ID: <4B4A546E.8010808@voidspace.org.uk>

On 10/01/2010 21:52, Dirkjan Ochtman wrote:
> On Sun, Jan 10, 2010 at 21:25, Brett Cannon<brett at python.org>  wrote:
>    
>> Michael has given me the hg transition/stdlib time slot at the language
>> summit this year. In regards to that I plan to lead a discussion on:
>> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
>> blog post on this topic recently:
>> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
>>      
> Unfortunately my flight doesn't land until about the time the
> Mercurial slot ends, so while I'll be there later on that afternoon
> for sprinting or questions or anything, I won't be at the actual
> Mercurial talk at the summit.
>
>    
We will probably be in 'open discussion' when you arrive so it will 
still be good to hear from you on the topic and there maybe questions 
that can't be answered until you arrive... :-)

Michael

> Cheers,
>
> Dirkjan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Mon Jan 11 00:02:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:02:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A5C69.7090007@v.loewis.de>

> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

I think it's more difficult than that. It also takes developer effort
to *commit* to the trunk. So if the trunk continued to live, would
it still be ok if I closed a 2.x feature request as "won't fix", or
only committed the new feature to the 3.x branch?

As for old branches: they *don't* live in the way you claim (i.e.
remain open with changes potentially just not released). Instead,
at some point, they are frozen to bug-fix only, then to security-fix
only, and then to no change at all.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 00:07:58 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:07:58 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A373F.9050601@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>
Message-ID: <4B4A5DCE.3070909@v.loewis.de>

> The announcement is precisely to avoid the situation where people commit
> new features to the 2.x main line of development (after the 2.7
> maintenance branch is created) in the expectation that they will be
> released as part of a hypothetical 2.8 release.
> 
> Whether that info needs to be in each and every 2.7 announcement...
> probably not. It isn't really info for users of Python, just for
> developers of Python.

I think the announcement is indeed also for the users, and I would
prefer it to stay in the release announcements, so that people will
know for sure (and speak up) before the 2.7 release happens.

As for decisions: I don't think there was an official BDFL pronouncent,
but I recall Guido posting a message close to that, proposing that 2.7
will be a release that will see bug fix releases for an indefinite
period of time (where indefinite != infinite). This was shortly after
him proposing that perhaps we shouldn't make a 2.7 release at all, and
stop at 2.6.

As for such a decision giving a bad light on Python: I don't think that
will be the case. Instead, I hear many users surprised for how long
we have been maintaining to parallel versions - "that must have taken
a lot of effort". So everybody will likely understand that enough is
enough.

Regards,
Martin


From benjamin at python.org  Mon Jan 11 01:07:35 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 18:07:35 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>

Consider this program:

class Descr(object):
    def __init__(self, name):
        self.name = name
    def __set__(self, instance, what):
        instance.__dict__[self.name] = what

class X(object):
    attr = Descr("attr")

x = X()
print(x.attr)
x.attr = 42
print(x.attr)

It gives in output:

<__main__.Descr object at 0x7fe1c9b28150>
42

The documentation [1] says that Descr is a data descriptor because it
defines the __set__ method. It also states that data descriptors
always override the value in the instance dictionary. So, the second
line should also be the descriptor object according to the
documentation.

My question is: Is this a doc bug or a implementation bug? If the
former, it will be the description of a data descriptor much less
consistent, since it will require that a __get__ method be present,
too. If the latter, the fix may break some programs relying on the
ability to "cache" a value in the instance dictionary.

[1] http://docs.python.org/reference/datamodel#invoking-descriptors


-- 
Regards,
Benjamin


From amauryfa at gmail.com  Mon Jan 11 01:51:09 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 11 Jan 2010 01:51:09 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>

Hi,

2010/1/11 Benjamin Peterson <benjamin at python.org>:
> Consider this program:
>
> class Descr(object):
> ? ?def __init__(self, name):
> ? ? ? ?self.name = name
> ? ?def __set__(self, instance, what):
> ? ? ? ?instance.__dict__[self.name] = what
>
> class X(object):
> ? ?attr = Descr("attr")
>
> x = X()
> print(x.attr)
> x.attr = 42
> print(x.attr)
>
> It gives in output:
>
> <__main__.Descr object at 0x7fe1c9b28150>
> 42
>
> The documentation [1] says that Descr is a data descriptor because it
> defines the __set__ method. It also states that data descriptors
> always override the value in the instance dictionary. So, the second
> line should also be the descriptor object according to the
> documentation.
>
> My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
>
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

Quoting the documentation:
"""Normally, data descriptors define both __get__() and __set__(),
while non-data descriptors have just the __get__() method.
"""
Your example is neither a data descriptor nor a non-data descriptor...

The thing that worries me a bit is the "x.attr" returning the Descr
object. Descriptors should remain at the class level, and instance
should only see values. I'd prefer an AttributeError in this case.

-- 
Amaury Forgeot d'Arc


From benjamin at python.org  Mon Jan 11 02:00:32 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 19:00:32 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
Message-ID: <1afaf6161001101700u21006708l2ef62bfa257e4ce7@mail.gmail.com>

2010/1/10 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...

See the footnote: http://docs.python.org/reference/datamodel#id7

>
> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.

Far too late for that, I'm afraid.



-- 
Regards,
Benjamin


From nas at arctrix.com  Mon Jan 11 02:44:57 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:44:57 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
Message-ID: <20100111014457.GA5289@arctrix.com>

On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> I don't think ending the 2.x series at 2.7 makes it look bad
> compared to 3.2; it's simply the end of a development line like
> any other software project. I suspect 2.7 will have a protracted
> bugfix window because so much code runs on 2.x exclusively at the
> moment.

I would guess over 99% of all Python code written doesn't run on
Python 3.  Given that, I think it is premature to close the door on
new major versions of Python 2.x. Also, we as a project should be
careful not to present the image that Python 2.x will not be
supported in the future.

> If there really is an outcry on this we can re-visit the issue,
> but as of right now we need to move forward at some point and 2.7
> seems like that good point.

I think that's bad PR.  If I had a successful product, I would not
announce its end of life just to see how many customers scream and
then decide if I should devote more resources to continue
maintaining it.

IMHO, the release notes should say something like:

     After the Python 2.7 release, the focus of Python development
     will be on Python 3.  There will continue to be maintainance
     releases of Python 2.x.


trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil


From tjreedy at udel.edu  Mon Jan 11 03:26:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Jan 2010 21:26:34 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <hie28p$joo$1@ger.gmane.org>

On 1/10/2010 8:44 PM, Neil Schemenauer wrote:
> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
>> I don't think ending the 2.x series at 2.7 makes it look bad
>> compared to 3.2; it's simply the end of a development line like
>> any other software project. I suspect 2.7 will have a protracted
>> bugfix window because so much code runs on 2.x exclusively at the
>> moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.

If the removal of old features had been done in the 2.x series, as once 
planned (Guido originally proposed removing the old meaning of int / int 
in 2.5) the same more or less would be true of 2.7. It is past time for 
other old and now duplicated features to be removed also.

   Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>
>> If there really is an outcry on this we can re-visit the issue,
>> but as of right now we need to move forward at some point and 2.7
>> seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.

Python is not being ended, but upgraded (with bloating duplications 
removed). Think of 3.1 as 2.8 with a new name.

tjr



From lrekucki at gmail.com  Mon Jan 11 04:26:48 2010
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Mon, 11 Jan 2010 04:26:48 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <dfbefb091001101926l366043f8mb95f196ba14f9c9@mail.gmail.com>

> Date: Mon, 11 Jan 2010 01:51:09 +0100
> From: "Amaury Forgeot d'Arc" <amauryfa at gmail.com>
> To: Benjamin Peterson <benjamin at python.org>
> Cc: Python Dev <python-dev at python.org>
> Subject: Re: [Python-Dev] Data descriptor doc/implementation
> ? ? ? ?inconsistency
> Message-ID:
> ? ? ? ?<e27efe131001101651y68e1da25je2a8d02f5c62ef19 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> 2010/1/11 Benjamin Peterson <benjamin at python.org>:
>> Consider this program:
>>
>> class Descr(object):
>> ? ?def __init__(self, name):
>> ? ? ? ?self.name = name
>> ? ?def __set__(self, instance, what):
>> ? ? ? ?instance.__dict__[self.name] = what
>>
>> class X(object):
>> ? ?attr = Descr("attr")
>>
>> x = X()
>> print(x.attr)
>> x.attr = 42
>> print(x.attr)
>>
>> It gives in output:
>>
>> <__main__.Descr object at 0x7fe1c9b28150>
>> 42
>>
>> The documentation [1] says that Descr is a data descriptor because it
>> defines the __set__ method. It also states that data descriptors
>> always override the value in the instance dictionary. So, the second
>> line should also be the descriptor object according to the
>> documentation.
>>
>> My question is: Is this a doc bug or a implementation bug? If the
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...
Actually, there is this footnote [1]:

""" A descriptor can define any combination of __get__(), __set__()
and __delete__(). If it does not define __get__(), then accessing the
attribute even on an instance will return the descriptor object
itself. If the descriptor defines __set__() and/or __delete__(), it is
a data descriptor; if it defines neither, it is a non-data descriptor.
"""

Which would mean Descr is actually a data descriptor without a
__get__(), so x.attr should always return the descriptor object itself
(at least in the docs).

> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.
>
> --
> Amaury Forgeot d'Arc

[1] http://docs.python.org/reference/datamodel#id7
-- 
Lukasz Rekucki


From brett at python.org  Mon Jan 11 04:46:04 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 19:46:04 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
Message-ID: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>

On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> > I don't think ending the 2.x series at 2.7 makes it look bad
> > compared to 3.2; it's simply the end of a development line like
> > any other software project. I suspect 2.7 will have a protracted
> > bugfix window because so much code runs on 2.x exclusively at the
> > moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x.


Well yeah; Python 3.0 is just over a year old with 3.1 -- the first robust
3.x release - is about six months old. From the beginning uptake was
expected to take years, not months, so saying that 3.x is not popular enough
seems premature.


> Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>

No one has said bugfixes will cease.


>
> > If there really is an outcry on this we can re-visit the issue,
> > but as of right now we need to move forward at some point and 2.7
> > seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.
>
>
I never said that I wanted to make this announcement specifically to provoke
outcries. I said if there happened to be outcries we could possibly visit
the issue again. Basically I was being considerate and trying to leave the
door open to discuss things in the future even though I don't see the
situation changing.


> IMHO, the release notes should say something like:
>
>     After the Python 2.7 release, the focus of Python development
>     will be on Python 3.  There will continue to be maintainance
>     releases of Python 2.x.
>

No because that suggests new features will be coming to 2.x which is not
going to happen. If you want to say there will be continual bugfix releases
for 2.7 as is par the course for Python and that the number of bugfix
releases might be more than normal then I am okay with that.


>
>
> trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil
>

That came and went already a couple months ago when we discussed stopping at
2.6 instead of 2.7 (see the various threads at
http://mail.python.org/pipermail/python-dev/2009-November/thread.html).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db586677/attachment-0005.htm>

From nas at arctrix.com  Mon Jan 11 05:05:07 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:05:07 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
Message-ID: <20100111040507.GA7200@arctrix.com>

On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> >     After the Python 2.7 release, the focus of Python development
> >     will be on Python 3.  There will continue to be maintainance
> >     releases of Python 2.x.
> 
> No because that suggests new features will be coming to 2.x which is not
> going to happen. If you want to say there will be continual bugfix releases
> for 2.7 as is par the course for Python and that the number of bugfix
> releases might be more than normal then I am okay with that.

Are you are saying that if someone steps up to merge the Unladen
Swallow features into a 2.8 release and someone also steps up to cut
the release that they will be prevented from doing so?

Also, are you presuming to channel the BDFL or was this dicussed
somewhere other than python-dev? Maybe I'm overreacting but I get
the feeling that the larger and less active segment of the Python
develpment team hasn't been involved in these decisions.

Best regards,

  Neil


From nas at arctrix.com  Mon Jan 11 05:17:54 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:17:54 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5C69.7090007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
Message-ID: <20100111041754.GB7200@arctrix.com>

On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
> [...] would it still be ok if I closed a 2.x feature request as
> "won't fix", or only committed the new feature to the 3.x branch?

Yes.  Non-bugfix development on 2.x would optional (i.e. done by
people who want to spend the time). Since language changes are now
out (an initiative I completely support), the majority of non-bugfix
changes would be optimizations and platform porting.

Regards,

  Neil


From brett at python.org  Mon Jan 11 06:06:15 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 21:06:15 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111040507.GA7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
Message-ID: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>

On Sun, Jan 10, 2010 at 20:05, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> > On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> > >     After the Python 2.7 release, the focus of Python development
> > >     will be on Python 3.  There will continue to be maintainance
> > >     releases of Python 2.x.
> >
> > No because that suggests new features will be coming to 2.x which is not
> > going to happen. If you want to say there will be continual bugfix
> releases
> > for 2.7 as is par the course for Python and that the number of bugfix
> > releases might be more than normal then I am okay with that.
>
> Are you are saying that if someone steps up to merge the Unladen
> Swallow features into a 2.8 release and someone also steps up to cut
> the release that they will be prevented from doing so?
>
>
I honestly don't know, but it's a possibility just like any other new
feature request. If people start taking the carrots we have added to 3.x and
backporting them to keep the 2.x series alive you are essentially making the
3.x DOA by negating its benefits which I personally don't agree with.


> Also, are you presuming to channel the BDFL or was this dicussed
> somewhere other than python-dev?


This was discussed; see the November 2009 threads. Guido actually suggested
ending the 2.x branch at 2.6 until people spoke up about the amount of stuff
already backported to 2.7 from 3.x because it was being assumed to be the
end of the series to warrant keeping it to help transition to 2.7.


> Maybe I'm overreacting but I get
> the feeling that the larger and less active segment of the Python
> develpment team hasn't been involved in these decisions.
>

This all came up in November from the 3rd through the 6th (four days) over a
ton of email traffic. This was not a snap decision but a heated discussion
where even Guido participated that culminated with the decision to stop at
2.7 for the 2.x series; this was not a smoked-filled room decision. I'm
sorry if people missed it when they weren't looking, but python-dev is
supposed to be the place to watch for this kind of stuff. I don't know how
else to bring this stuff to people's attention short of also on
python-committers, but developers are basically expected to be on both lists
so I don't see where anyone did anything wrong in terms of informing
developers.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/18c3bcd7/attachment-0005.htm>

From nas at arctrix.com  Mon Jan 11 06:27:13 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 23:27:13 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
Message-ID: <20100111052713.GA8211@arctrix.com>

On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> If people start taking the carrots we have added to 3.x and
> backporting them to keep the 2.x series alive you are essentially
> making the 3.x DOA by negating its benefits which I personally
> don't agree with.

I think we have got to the heart of our disagreement. Assume that
some superhuman takes all the backwards compatible goodies from 3.x
and merges them into 2.x. If that really would kill off Python 3
then I would proclaim it a project that deserved to die. Why should
the backwards incompatible changes be endured if they cannot justify
their existance by the benefit they provide?

I guess I have more confidence in Python 3 than you do. I don't see
why Python 2.x needs to be artificially limited so that Python 3 can
benefit.

Regards,

  Neil


From brett at python.org  Mon Jan 11 07:30:49 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 22:30:49 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
Message-ID: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:27, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> > If people start taking the carrots we have added to 3.x and
> > backporting them to keep the 2.x series alive you are essentially
> > making the 3.x DOA by negating its benefits which I personally
> > don't agree with.
>
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x. If that really would kill off Python 3
> then I would proclaim it a project that deserved to die. Why should
> the backwards incompatible changes be endured if they cannot justify
> their existance by the benefit they provide?
>
>
When I said "carrots" I meant everything that makes Python 3 the best
version of Python, including the backward-incompatible changes.

I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.
>

I don't view it as artificial but simply a focusing of the development team
on what has been determined as the future of Python by Guido and letting go
of the past.

IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev
saying "Python 3 is the future, but we are keeping the old Python 2.x around
because we don't have *that* much faith in the future we have laid out".
That's poison to Python 3 by showing a lack of confidence in the direction
that the BDFL and python-dev as a group has chosen. Now I could be wrong and
there could actually be a large number of active contributors who want to
keep the 2.x series going, but based on the discussion that occurred the
last time this came up I believe the guys who are keeping Python running are
ready to move on.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/36448e5f/attachment-0005.htm>

From regebro at gmail.com  Mon Jan 11 08:08:14 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 08:08:14 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <319e029f1001102308p5de87cber2131f3aec1abc9f0@mail.gmail.com>

On Mon, Jan 11, 2010 at 06:27, Neil Schemenauer <nas at arctrix.com> wrote:
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x.

The superhumans of core developers pretty much already have. That's
pretty much 2.7 you are talking about. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From chrism at plope.com  Mon Jan 11 08:27:03 2010
From: chrism at plope.com (Chris McDonough)
Date: Mon, 11 Jan 2010 02:27:03 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
Message-ID: <4B4AD2C7.1050703@plope.com>

Brett Cannon wrote:
> IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev 
> saying "Python 3 is the future, but we are keeping the old Python 2.x 
> around because we don't have *that* much faith in the future we have 
> laid out". That's poison to Python 3 by showing a lack of confidence in 
> the direction that the BDFL and python-dev as a group has chosen. Now I 
> could be wrong and there could actually be a large number of active 
> contributors who want to keep the 2.x series going, but based on the 
> discussion that occurred the last time this came up I believe the guys 
> who are keeping Python running are ready to move on.

I think this is mostly just a branding issue.  Once the folks who have 
historically kept Python 2 running really *do* move on, there will be other 
folks who will step up and become maintainers out of necessity: those stuck on 
old platforms, permanent stragglers, people who for whatever reason actually 
like Python 2 better, etc.  It's just going to happen, there's really nothing 
anybody can do about it.

I don't think there's anything wrong with this.  If such a group of folks wants 
to get together and create another Python 2.X release, there's nothing stopping 
them from doing so except the above branding declaration.  I'd personally not 
close the door on allowing these folks to call such an effort "Python 2".

- C



From martin at v.loewis.de  Mon Jan 11 09:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:06:16 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
	<20100111041754.GB7200@arctrix.com>
Message-ID: <4B4ADBF8.3030803@v.loewis.de>

Neil Schemenauer wrote:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time). 

I think *that* would give a very bad impression of Python. Depending
whom you deal with, the new feature you want may or may not get
added to the code base. Contributors would feel even more stranded
than they do now, where it may take several years to get a patch
reviewed, as you then could submit a patch, and pray that a comitter
reviews it who believes in future 2.x releases.

The point of setting policies is that it gives every user (contributors,
committers, and "end-user" developers) a reliable foundation for
expectations.

Regards,
Martin


From arcriley at gmail.com  Mon Jan 11 09:06:10 2010
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 11 Jan 2010 03:06:10 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AD2C7.1050703@plope.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com> 
	<4B4AD2C7.1050703@plope.com>
Message-ID: <bad82a81001110006n3cacd953g98fb25e96330ee89@mail.gmail.com>

after all these years, some people still run AmigaOS.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/4fdcbb4e/attachment-0005.htm>

From martin at v.loewis.de  Mon Jan 11 09:11:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:11:25 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <4B4ADD2D.6070906@v.loewis.de>

> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.

Why that? It is a fact: 2.x will not be supported, in some future.
Should we lie to users?

>      After the Python 2.7 release, the focus of Python development
>      will be on Python 3.  There will continue to be maintainance
>      releases of Python 2.x.

It shouldn't say that, because it wouldn't be true.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 09:18:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:18:30 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <4B4ADED6.5080207@v.loewis.de>

> I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.

Because it takes too much time. Too much of my time, but apparently
also too much of other people's time.

Of course, the less active fraction of Python contributors may not
notice, since they just chose to not contribute (which, of course,
is fine). However, asking me to work twice as much as I want to
on the project to keep two branches alive is just unfair.

This has nothing to do with pushing 3.x, but all with managing
available manpower and still providing quality software.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 09:42:51 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 09:42:51 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>

This is what it says now:

"Python 2.7 is scheduled to be the last major version in the 2.x
series before it moves into 5 years of bugfix-only mode. "

And during this discussion it has been noted that others than the core
python team might pick up Python 2 and make releases. So maybe we can
and this discussion by changing that line in future releases to be:

"Python 2.7 is scheduled to be the last major version in the 2.x
series released by the Python Software Foundation before it moves into
5 years of bugfix-only mode. "

That doesn't exclude *others* making a Python 2.8 that could include
all sorts of crazy features. :)

This is mainly just to get the pointless discussion over with. The
current Python core team don't want to make a 2.8, so that will not
happen. Someone else might, but as Chris points out, they won't step
up until they have to, which is probably at least two years from now.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 10:06:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 10:06:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
Message-ID: <4B4AEA25.7010100@v.loewis.de>

> "Python 2.7 is scheduled to be the last major version in the 2.x
> series released by the Python Software Foundation before it moves into
> 5 years of bugfix-only mode. "
> 
> That doesn't exclude *others* making a Python 2.8 that could include
> all sorts of crazy features. :)
> 
> This is mainly just to get the pointless discussion over with.

I know you are just looking for a compromise, but this shouldn't be it:
the PSF has deliberately stayed out of the actual Python engineering,
so the release that Benjamin makes is not done by the PSF (but by
Benjamin and his contributors :-).

I like the wording as it is (although I find the promise of five years
of bug fix releases a bit too strong).

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 10:19:24 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 10:19:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AEA25.7010100@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
Message-ID: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>

On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I know you are just looking for a compromise, but this shouldn't be it:
> the PSF has deliberately stayed out of the actual Python engineering,
> so the release that Benjamin makes is not done by the PSF (but by
> Benjamin and his contributors :-).

Hm. Yeah. That's right of course. I started with saying "official",
but then I thought "official according to who?". :) So I changed it to
mentioning PSF, but that doesn't work either. I guess the current
writing as as good as it gets, unless we change "scheduled" to
"expected" or something.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Mon Jan 11 10:42:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 Jan 2010 10:42:05 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A5C69.7090007@v.loewis.de> <20100111041754.GB7200@arctrix.com>
Message-ID: <hierpd$dm1$1@ger.gmane.org>

Neil Schemenauer, 11.01.2010 05:17:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time).

Note that there's also the time it takes to make a release (usually 
involving at least one beta release), plus the possibility of introducing 
bugs while adding new features or even while fixing agreed bugs. All of 
this takes time in addition to the time people might want to invest in 
'real' development on the 2.x trunk.

Stefan



From stephen at xemacs.org  Mon Jan 11 10:59:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 11 Jan 2010 18:59:57 +0900
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <8763793qsi.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Schemenauer writes:
 > On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:

 > > If people start taking the carrots we have added to 3.x and
 > > backporting them to keep the 2.x series alive you are essentially
 > > making the 3.x DOA by negating its benefits which I personally
 > > don't agree with.

Well, I think it's *worse* than that, and I don't think you really
mean "DOA", anyway.  (Feel free to correct me, of course.)

The problem I see with backporting lots of stuff, and/or adding new
features that aren't in 3.0, to 2.x is that it will make 2.x even
cruftier, when it was already crufty enough that Guido (and almost all
of python-dev) bit the bullet and said "backward compatibility is no
excuse for keeping something in 3.0".

That surely means that a lot of python-dev denizens will declare
non-support 2.x for x > 7.  It's not going to be the gradual migration
we've seen over the past few months as active people start to spend
more and more time on 3 vs. 2; it will be a watershed.  Especially if
these are new features merged from outside that the "small active
segment" doesn't know anything about.  From the users' point of view,
that amounts to a *fork*, even if it's internal and "friendly".

 > I think we have got to the heart of our disagreement. Assume that
 > some superhuman takes all the backwards compatible goodies from 3.x
 > and merges them into 2.x.

Isn't that a bit ridiculous?  I just don't see any evidence that
anything like that is going to happen.  Worse, if we *assume* it will
happen, I don't see any way to assess whether (1) Python 3 goes belly
up, (2) there's an effective fork confusing the users and draining the
energy of python-dev, or (3) everybody goes "wow!" because it's so
cool that everybody wants to keep maintaining an extra 3 branches
indefinitely.

My opinion is that given the clear direction the "small active
segment" is going, telling the users anything but what Brett proposed
is disinformation.

 > I guess I have more confidence in Python 3 than you do. I don't see
 > why Python 2.x needs to be artificially limited so that Python 3 can
 > benefit.

It's not for Python 3, which you, I, and I'm pretty sure Brett-in-his-
heart-of-hearts agree can take care of itself because it is *better*
than Python 2.  It's for Python, and for the Python community.


From walter at livinglogic.de  Mon Jan 11 11:37:56 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 11:37:56 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001091438.43576.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
Message-ID: <4B4AFF84.7070206@livinglogic.de>

On 09.01.10 14:38, Victor Stinner wrote:

> Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
>>> Good idea, I choosed open(filename, encoding="BOM").
>>
>> On the surface this looks like there's an encoding named "BOM", but
>> looking at your patch I found that the check is still done in
>> TextIOWrapper. IMHO the best approach would to the implement a *real*
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
>> the IO library. It could even be developed as a standalone project and
>> published in the Cheeseshop.
> 
> Why not, this is another solution to the point (2) (Check for a BOM while 
> reading or detect it before?). Which encoding would be used if there is not 
> BOM? UTF-8 sounds like a good choice.

UTF-8 might be a good choice, are the failback could be specified in the
encoding name, i.e.

   open("file.txt", encoding="BOM-UTF-8")

falls back to UTF-8, if there's no BOM at the start.

This could be implemented via a custom codec search function (see
http://docs.python.org/library/codecs.html#codecs.register for more info).

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 12:12:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:12:20 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4AFF84.7070206@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
Message-ID: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>

On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
> UTF-8 might be a good choice

No, fallback if there is no BOM should be the local settings, just as
fallback is today if you don't specify a codec.
I mean, what if you want to look for a BOM but fall back to something
else? How far will we go with encoding special information in the
codecs names? codec='BOM else UTF-16 else locale'? :-)

BOM is not a locale, and should not be a locale. Having a locale
called BOM is wrong per se. It should either be default to look for a
BOM when codec=None, or a special parameter. If none of these are
desired, then we need a special function that takes a filename or file
handle, and looks for a BOM and returns the codec found or None. But
I find that much less natural and obvious than checking the BOM when codec=None.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From regebro at gmail.com  Mon Jan 11 12:13:00 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:13:00 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <319e029f1001110313u1d839deodfe06a6c7ec423cf@mail.gmail.com>

On Mon, Jan 11, 2010 at 12:12, Lennart Regebro <regebro at gmail.com> wrote:
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se.

D'oh! I mean codec here obviously.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 13:29:04 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 13:29:04 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4913DF.7050801@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>
Message-ID: <4B4B1990.90705@livinglogic.de>

On 10.01.10 00:40, "Martin v. L?wis" wrote:
>>> How does the requirement that it be implemented as a codec miss the
>>> point?
>>
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
> 
> Yes - however, Victor currently apparently *doesn't* want it to be the
> default, but wants the user to specify encoding="BOM". If so, it isn't
> the default, and it is easy to implement as a codec.
> 
>>> FWIW, I agree with Walter that if it is provided through the encoding=
>>> argument, it should be a codec. If it is built into the open function
>>> (for whatever reason), it must be provided by some other parameter.
>>
>> Why not simply encoding=None?
> 
> I don't mind. Please re-read Walter's message - it only said that
> *if* this is activated through encoding="BOM", *then* it must be
> a codec, and could be on PyPI. I don't think Walter was talking about
> the case "it is not activated through encoding='BOM'" *at all*.

However if this autodetection feature is useful in other cases (no
matter how it's activated), it should be a codec, because as part of the
open() function it isn't reusable.

>> The default value should provide the most useful
>> behaviour possible. Forcing users to choose between two different autodetection
>> strategies (encoding=None and another one) is a little insane IMO.

And encoding="mbcs" is a third option on Windows.

> That wouldn't disturb me much. There are a lot of things in that area
> that are a little insane, starting with Microsoft Windows :-)

;)

Servus,
   Walter


From barry at python.org  Mon Jan 11 13:37:46 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 07:37:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5DCE.3070909@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com> <4B4A5DCE.3070909@v.loewis.de>
Message-ID: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>

On Jan 10, 2010, at 6:07 PM, Martin v. L?wis wrote:

> As for decisions: I don't think there was an official BDFL pronouncent,
> but I recall Guido posting a message close to that, proposing that 2.7
> will be a release that will see bug fix releases for an indefinite
> period of time (where indefinite != infinite). This was shortly after
> him proposing that perhaps we shouldn't make a 2.7 release at all, and
> stop at 2.6.

IMO, the focus for Python 2.7 (and beyond) must be on helping people transition to Python 3.  That should be the reason why a 2.7 is released and, what should dictate whether a 2.8 is necessary.

Maybe everything people need (except manpower and round tuits) is already there.  I was pleasantly surprised to find that Distribute supports automatically running 2to3 at install time for Python packages.  This allowed me to "port" a recent (admittedly small) package to Python 3 by making it "python2.6 -3" clean and adding a couple of attributes to my setup.py[1].  I don't know how widely that feature's known, but I think it's *huge*, and exactly what we need to be doing.  The more we can make it easy for people to port their Python 2 code to Python 3, and to support that with one code base, the quicker we'll get to a predominantly Python 3 world.

So the question we should be asking is: what's missing from Python 2.7 to help with this transition?  If we can't get it into 2.7, then why, and would pushing it back to 2.8 help at all?

-Barry

[1] modulo a bug in Distribute that caused doctest in separate files to not be used when running  under Python 3.



From solipsis at pitrou.net  Mon Jan 11 13:39:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 13:39:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>  <4B4B1990.90705@livinglogic.de>
Message-ID: <1263213574.3327.0.camel@localhost>


> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

It is reusable as part of io.TextIOWrapper, though.

Regards

Antoine.




From regebro at gmail.com  Mon Jan 11 13:45:30 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 13:45:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
Message-ID: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>

On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

But an autodetect feature is not a codec. Sure it should be reusable,
but making it a codec seems to be  a weird hack to me. And how would
you reuse it if it was a codec? A reusable autodetect feature would be
useable to detect what codec it is. A autodetect codec would not be
useful for that, as it would simply just decode.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 14:21:15 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 14:21:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>	<4B4913DF.7050801@v.loewis.de>
	<4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B25CB.5030003@livinglogic.de>

On 11.01.10 13:45, Lennart Regebro wrote:

> On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
>> However if this autodetection feature is useful in other cases (no
>> matter how it's activated), it should be a codec, because as part of the
>> open() function it isn't reusable.
> 
> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

I think we already had this discussion two years ago in the context of
XML decoding ;):

http://mail.python.org/pipermail/python-dev/2007-November/075138.html

> And how would
> you reuse it if it was a codec? A reusable autodetect feature would be
> useable to detect what codec it is. A autodetect codec would not be
> useful for that, as it would simply just decode.

I have implemented an XML codec (as part of XIST:
http://pypi.python.org/pypi/ll-xist), that can do that:

>>> from ll import xml_codec
>>> import codecs
>>> c = codecs.getincrementaldecoder("xml")()
>>> c.encoding
>>> c.decode("<?xml")
u''
>>> c.encoding
>>> c.decode(" version='1.0'")
u''
>>> c.encoding
>>> c.decode(" encoding='iso-8859-1'?>")
u"<?xml version='1.0' encoding='iso-8859-1'?>"
>>> c.encoding
'iso-8859-1'

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 14:47:12 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 14:47:12 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B25CB.5030003@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B25CB.5030003@livinglogic.de>
Message-ID: <319e029f1001110547n1d1c9831p34b0eac519d13f9d@mail.gmail.com>

On Mon, Jan 11, 2010 at 14:21, Walter D?rwald <walter at livinglogic.de> wrote:
> I think we already had this discussion two years ago in the context of
> XML decoding ;):

Yup. Ans Martins answer then is my answer now:

"> So the code is good, if it is inside an XML parser, and it's bad if it
> is inside a codec?

Exactly so. This functionality just *isn't* a codec - there is no
encoding. Instead, it is an algorithm for *detecting* an encoding."

The conclusion was that a method do autodetect encodings would be
good. I think the same conclusion applies here.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 18:16:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:16:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<4B46F67F.60604@v.loewis.de>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B5CF5.50806@v.loewis.de>

> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

Well, the existing UTF-16 codec also is an autodetect feature (to
detect the endianness), and I don't consider it a weird hack.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 18:27:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:27:01 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B5CF5.50806@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B5CF5.50806@v.loewis.de>
Message-ID: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:16, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> But an autodetect feature is not a codec. Sure it should be reusable,
>> but making it a codec seems to be ?a weird hack to me.
>
> Well, the existing UTF-16 codec also is an autodetect feature (to
> detect the endianness), and I don't consider it a weird hack.

So the BOM codec should raise a UnicodeDecodeError if there is no BOM?
Because that's what it would have to do, in that case, because it
can't fall back on anything, it has to handle and implement all
encodings that have a BOM. And is it then actually very useful? You
would have to do a try/except first with encoding='BOM' and then
encoding=None to get the fallback to the standard.


I must say that I find this whole thing pretty obvious. 'BOM' is not
an encoding. Either there should be a method to get the encoding from
the BOM, returning None of there isn't one, or open() should look at
the BOM when you pass in encoding=None. Or both.

That covers all usecases, is easy and obvious. Either open(file=foo,
encoding=None) or open(file, encoding=encoding_from_bom(file))

I can't see that open(file, encoding='BOM') has any benefit over this,
covers any extra usecase and is clearer in any way. Instead it adds
something confusing: An encoding that isn't an encoding.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From python at mrabarnett.plus.com  Mon Jan 11 18:35:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:35:35 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<201001091438.43576.victor.stinner@haypocalc.com>	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <4B4B6167.40606@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
>> UTF-8 might be a good choice
> 
> No, fallback if there is no BOM should be the local settings, just as
> fallback is today if you don't specify a codec.
> I mean, what if you want to look for a BOM but fall back to something
> else? How far will we go with encoding special information in the
> codecs names? codec='BOM else UTF-16 else locale'? :-)
> 
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se. It should either be default to look for a
> BOM when codec=None, or a special parameter. If none of these are
> desired, then we need a special function that takes a filename or file
> handle, and looks for a BOM and returns the codec found or None. But
> I find that much less natural and obvious than checking the BOM when codec=None.
> 
Or pass a function that accepts a byte stream or the first few bytes and
returns the encoding and any unused bytes (because the byte stream might
not be seekable)?

def guess_encoding(byte_stream):
     data = byte_stream.read(2)
     if data == b"\xFE\xFF":
         return "UTF-16BE", b""
     return "UTF-8", data

text_file = open(filename, encoding=guess_encoding)
...


From python at mrabarnett.plus.com  Mon Jan 11 18:46:30 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:46:30 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
Message-ID: <4B4B63F6.1020309@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>> I know you are just looking for a compromise, but this shouldn't be
>> it: the PSF has deliberately stayed out of the actual Python
>> engineering, so the release that Benjamin makes is not done by the
>> PSF (but by Benjamin and his contributors :-).
> 
> Hm. Yeah. That's right of course. I started with saying "official", 
> but then I thought "official according to who?". :)

"Official" is whatever the BDFL says it is! :-)

> So I changed it to mentioning PSF, but that doesn't work either. I
> guess the current writing as as good as it gets, unless we change
> "scheduled" to "expected" or something.
> 


From martin at v.loewis.de  Mon Jan 11 18:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:59:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>	
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>	
	<4B4B5CF5.50806@v.loewis.de>
	<319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
Message-ID: <4B4B66EC.7000203@v.loewis.de>

> I must say that I find this whole thing pretty obvious. 'BOM' is not
> an encoding.

That I certainly agree with.

> That covers all usecases, is easy and obvious. Either open(file=foo,
> encoding=None) or open(file, encoding=encoding_from_bom(file))
> 
> I can't see that open(file, encoding='BOM') has any benefit over this,

Well, it would have the advantage that Walter pointed out: you can
implement it independent of the open() implementation, and even provide
it in older versions of Python.

Regards,
Martin



From regebro at gmail.com  Mon Jan 11 18:59:52 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:59:52 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B63F6.1020309@mrabarnett.plus.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
Message-ID: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:46, MRAB <python at mrabarnett.plus.com> wrote:
> "Official" is whatever the BDFL says it is! :-)

Heh, right.

So, it should say "Guido wants 2.7 to be the last main version of
Python 2, so it probably will be. We promise to release bugfixes it
for, like, ages".

No need to be needlessly formal. :-D
-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From olemis at gmail.com  Mon Jan 11 19:58:01 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 13:58:01 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
[...]
>

I had similar issues too (please read below ;o) ...

On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>

About guessing the encoding, I experienced this issue while I was
developing a Trac plugin. What I was doing is as follows :

- I guessed the MIME type + charset encoding using Trac MIME API (it
was a CSV file encoded using UTF-16)
- I read the file using `open`
- Then wrapped the file using `codecs.EncodedFile`
- Then used `csv.reader`

... and still get the BOM in the first value of the first row in the CSV file.

{{{
#!python

>>> mimetype
'utf-16-le'
>>> ef = EncodedFile(f, 'utf-8', mimetype)
}}}

IMO I think I am +1 for leaving `open` just like it is, and use module
`codecs` to deal with encodings, but I am strongly -1 for returning
the BOM while using `EncodedFile` (mainly because encoding is
explicitly supplied in ;o)

> --Guido
>

CMIIW anyway ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:


From mal at egenix.com  Mon Jan 11 21:44:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 11 Jan 2010 21:44:34 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
Message-ID: <4B4B8DB2.1060102@egenix.com>

Olemis Lang wrote:
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>> Hi,
>>>
>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>> BOM should be "ignored".
>>>
> [...]
>>
> 
> I had similar issues too (please read below ;o) ...
> 
> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
> 
> About guessing the encoding, I experienced this issue while I was
> developing a Trac plugin. What I was doing is as follows :
> 
> - I guessed the MIME type + charset encoding using Trac MIME API (it
> was a CSV file encoded using UTF-16)
> - I read the file using `open`
> - Then wrapped the file using `codecs.EncodedFile`
> - Then used `csv.reader`
> 
> ... and still get the BOM in the first value of the first row in the CSV file.

You didn't say, but I presume that the charset guessing logic
returned either 'utf-16-le' or 'utf-16-be' - those encodings don't
remove the leading BOM. The 'utf-16' codec will remove the BOM.

> {{{
> #!python
> 
>>>> mimetype
> 'utf-16-le'
>>>> ef = EncodedFile(f, 'utf-8', mimetype)
> }}}

Same here: the UTF-8 codec will not remove the BOM, you have
to use the 'utf-8-sig' codec for that.

> IMO I think I am +1 for leaving `open` just like it is, and use module
> `codecs` to deal with encodings, but I am strongly -1 for returning
> the BOM while using `EncodedFile` (mainly because encoding is
> explicitly supplied in ;o)

Note that EncodedFile() doesn't do any fancy BOM detection or
filtering. This is the job of the codecs.

Also note that BOM removal is only valid at the beginning of
a file. All subsequent BOM-bytes have to be read as-is (they
map to a zero-width non-breaking space) - without removing them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Mon Jan 11 22:12:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 07:12:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <4B4B9447.4060101@gmail.com>

Benjamin Peterson wrote:
 > My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
> 
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

I would call it a documentation bug: by leaving out the __get__ you're
telling Python to override *just* the setting of the attribute, while
still using the normal lookup process for retrieving the attribute (i.e.
allowing the attribute to be shadowed in the instance dictionary).

Adding a "print('Descriptor Invoked')" to the __set__ method in your
example:

>>> x = X()
>>> print(x.attr)
<__main__.Descr object at 0x7f283b51ac50>
>>> x.attr = 42
Descriptor invoked
>>> print(x.attr)
42
>>> x.attr = 6*9
Descriptor invoked
>>> print(x.attr)
54

Note that the behaviour here is still different from that of a data
descriptor: with a data descriptor, once it gets shadowed in the
instance dictionary, the descriptor is ignored *completely*. The only
way to get the descriptor involved again is to eliminate the shadowing.
The non-data descriptor with only __set__ is just choosing not to
override the attribute lookup process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From olemis at gmail.com  Mon Jan 11 22:29:38 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 16:29:38 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B8DB2.1060102@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
	<4B4B8DB2.1060102@egenix.com>
Message-ID: <24ea26601001111329i7f609aa1i9f5db7eb61f1fae7@mail.gmail.com>

Probably one part of this is OT , but I think it could complement the
discussion ;o)

On Mon, Jan 11, 2010 at 3:44 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Olemis Lang wrote:
>>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>>> <victor.stinner at haypocalc.com> wrote:
>>>> Hi,
>>>>
>>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>>> BOM should be "ignored".
>>>>
>> [...]
>>>
>>
>> I had similar issues too (please read below ;o) ...
>>
>> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>>>
>>
>> About guessing the encoding, I experienced this issue while I was
>> developing a Trac plugin. What I was doing is as follows :
>>
>> - I guessed the MIME type + charset encoding using Trac MIME API (it
>> was a CSV file encoded using UTF-16)
>> - I read the file using `open`
>> - Then wrapped the file using `codecs.EncodedFile`
>> - Then used `csv.reader`
>>
>> ... and still get the BOM in the first value of the first row in the CSV file.
>
> You didn't say, but I presume that the charset guessing logic
> returned either 'utf-16-le' or 'utf-16-be'

Yes. In fact they return the full mimetype 'text/csv; charset=utf-16-le' ... ;o)

> - those encodings don't
> remove the leading BOM.

... and they should ?

> The 'utf-16' codec will remove the BOM.
>

In this particular case there's nothing I can do, I have to process
whatever charset is detected in the input ;o)

>> {{{
>> #!python
>>
>>>>> mimetype
>> 'utf-16-le'
>>>>> ef = EncodedFile(f, 'utf-8', mimetype)
>> }}}
>
> Same here: the UTF-8 codec will not remove the BOM, you have
> to use the 'utf-8-sig' codec for that.
>
>> IMO I think I am +1 for leaving `open` just like it is, and use module
>> `codecs` to deal with encodings, but I am strongly -1 for returning
>> the BOM while using `EncodedFile` (mainly because encoding is
>> explicitly supplied in ;o)
>
> Note that EncodedFile() doesn't do any fancy BOM detection or
> filtering.

... directly.

> This is the job of the codecs.
>

OK ... to come back to the scope of the subject, in the general case,
I think that BOM (if any) should be handled by codecs, and therefore
indirectly by EncodedFile . If that's a explicit way of working with
encodings I'd prefer to use that wrapper explicitly in order to
(encode | decode) the file and let the codec detect whether there's a
BOM or not and ?adjust? `tell`, `read` and everything else in that
wrapper (instead of `open`).

> Also note that BOM removal is only valid at the beginning of
> a file. All subsequent BOM-bytes have to be read as-is (they
> map to a zero-width non-breaking space) - without removing them.
>

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Test cases for custom query (i.e report 9) ... PASS (1.0.0)  -
http://simelo.hg.sourceforge.net/hgweb/simelo/trac-gviz/rev/d276011e7297


From martin at v.loewis.de  Mon Jan 11 22:42:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:42:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
Message-ID: <4B4B9B55.1010709@v.loewis.de>

> So the question we should be asking is: what's missing from Python
> 2.7 to help with this transition?

Wrt. the distribute feature you've noticed: Python 3.0 already supports
that in distutils. So from day one of Python 3.x, you could have used
that approach for porting Python code. I had been promoting it ever
since, but it's taking up only slowly, partially because it has
competition in the approaches "burn the bridges" (i.e. convert to 3.x
once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
(i.e. try to write code that works in both 2.x and 3.x unmodified).

> If we can't get it into 2.7, then
> why, and would pushing it back to 2.8 help at all?

I've done a fair bit of 3.x porting, and I'm firmly convinced that
2.x can do nothing:

a) telling people that they have to move to 2.6 first actually
   hurts migration, instead of helping, because it implies to them
   that they have to drop old versions (e.g. 2.3.) - just because
   they had *always* dropped old versions before supporting new ones.
b) IMO, people also don't gain much by first migrating to 2.6.
   In principle, it gives them the opportunity to get py3k warnings.
   However, I haven't heard a single positive report where these
   warnings have actually helped people in porting. Yours is the
   first report saying that you followed the official guideline,
   but you didn't state whether doing so actually helped (or whether
   you just ported to 2.6 because the guideline told you to).
c) whatever 2.7 does (except perhaps for the warnings), it won't
   be useful to applications, because they couldn't use it, anyway:
   they need to support 2.4 and 2.5, and it won't have any of the
   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
   might actually hurt porting: people may have to special-case
   2.7 because their work-arounds for older versions may break in 2.7
   (e.g. testing whether a name is *not* defined, when it becomes
   defined in 2.7), plus it gives them an incentive to not port
   yet until they can drop support for anything before 2.7.

Inherently, 2.8 can't improve on that.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 22:44:24 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:44:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
Message-ID: <4B4B9BB8.3070505@v.loewis.de>

> So, it should say "Guido wants 2.7 to be the last main version of
> Python 2, so it probably will be. We promise to release bugfixes it
> for, like, ages".
> 
> No need to be needlessly formal. :-D

That summarizes my understanding of what Guido said fairly well :-)

+1 for putting it into the announcements.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Mon Jan 11 23:11:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 11 Jan 2010 22:11:44 +0000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4B9447.4060101@gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com>
Message-ID: <4B4BA220.20701@voidspace.org.uk>

On 11/01/2010 21:12, Nick Coghlan wrote:
> Benjamin Peterson wrote:
>   >  My question is: Is this a doc bug or a implementation bug? If the
>    
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>>      
> [snip...]
>
> Note that the behaviour here is still different from that of a data
> descriptor: with a data descriptor, once it gets shadowed in the
> instance dictionary, the descriptor is ignored *completely*. The only
> way to get the descriptor involved again is to eliminate the shadowing.
> The non-data descriptor with only __set__ is just choosing not to
> override the attribute lookup process.
>
>    

Does that mean we need a third class of descriptors that are neither 
data descriptors nor non-data descriptors?

What should we call them: really-not-data-descriptors?

All the best,

Michael

> Cheers,
> Nick.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From guido at python.org  Mon Jan 11 23:55:01 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Jan 2010 14:55:01 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9BB8.3070505@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de> 
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com> 
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com> 
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com> 
	<4B4B9BB8.3070505@v.loewis.de>
Message-ID: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>

+1 from me. (With the caveat that "time will tell" is definitely the
ultimate answer. Plans may change unexpectedly, even though we don't
expect them to.)

PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
helpful. But I trust he has ported a lot more code to 3.x than I have
personally. Are there other experiences?

--Guido

On Mon, Jan 11, 2010 at 1:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So, it should say "Guido wants 2.7 to be the last main version of
>> Python 2, so it probably will be. We promise to release bugfixes it
>> for, like, ages".
>>
>> No need to be needlessly formal. :-D
>
> That summarizes my understanding of what Guido said fairly well :-)
>
> +1 for putting it into the announcements.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Tue Jan 12 00:16:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 00:16:47 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <4B4BB15F.5020807@v.loewis.de>

Guido van Rossum wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
> 
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. 

That's really because I haven't even attempted to use it. Instead, the
software would fall flat on its own when running the test suite in 3.x.
IOW, I don't need 2.6 to tell me what might break when I can see 3.x
actually breaking.

Regards,
Martin


From david.lyon at preisshare.net  Tue Jan 12 00:22:42 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:22:42 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>


Hi Martin,

> Of course, the less active fraction of Python contributors may not
> notice, since they just chose to not contribute (which, of course,
> is fine). However, asking me to work twice as much as I want to
> on the project to keep two branches alive is just unfair.

Totally true. Actually as an end-developer I'd say that python
2.x series from a programming perspective is quite good. It
doesn't need the addition of a lot of new features imho.

So for me, I think too much time spent there would be not
yield great benefits.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Well, we all know your work is super quality. :-) That's not
being contested.

However, Quality can be measured different ways and it can
be assessed in different ways. Quality itself is a subjective
thing.

The point I'm only making is that if a piece of software
doesn't have "new" things added over time, then users
can get a reverse impression of a lack of quality.

We've all seen where 'internal' quality can increase
and user perceptions can decrease.

It could be things like improved graphics and things readily
apparent to the user.

At the moment, I would say that the "internal" quality
of the python 2.x series is super high. "external" quality
issues such as the packaging dilemma give the user the
opposite quality experience. But people are working on
that as best they can elsewhere. I'll leave it at that.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Python 3.x needs more carrots.

>From an ordinary (perphaps ignorant) user perspective there
is nothing. Yes, we know if we actually will start
programming then we will like it more.

But my wishes to Santa Claus would be allow the free
flow of PEPs for Python 3 packaging. Even encourage
it.

As an end developer, here's what I'd like from Santa
in 2010 to get me to swap to python 3:

 * get all the packages on pypi tested for python 3

 * put a web based package manager in python 3. This
   would perhaps be based around PIP (keep many
   people happy) and would look much like the built
   in web-console that you get with a $200 router.

 * Incorporate SCM based end-user software installs
   by adding to python3-stdlib the SCM support
   packages (cvs,bzr,svn,hg). This would *really*
   help in the scientific community.

 * put a web interface on distutils also so that
   we don't have to use a command line interface.
   (I want a pic of a smiley girl to great me
    when I build something - "Are you ready
    to build now Honey?"). ok - I joke. But the
    point is made.

So, ok, maybe these things aren't about 'code'
quality. But rather user experience.

Things like these do count also as "quality"
via the technical term "perception of quality".

If the PEP process is as unblocked as the
documentation implies, implying that anybody
can contribute to Python 3. Then there shouldn't
be any issue.

David









From solipsis at pitrou.net  Tue Jan 12 00:37:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 23:37:47 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100112T003506-611@post.gmane.org>

David Lyon <david.lyon <at> preisshare.net> writes:
> 
> > This has nothing to do with pushing 3.x, but all with managing
> > available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As someone who experiences the difference almost every day, I can say 3.x
definitely has enough carrots for me. The simple fact that I will be able to
raise exceptions with non-ASCII error messages without having to workaround a
hopelessly broken conversion/reporting logic is a huge improvement. And that's
only a small part of the picture.

Regards

Antoine.




From janssen at parc.com  Tue Jan 12 00:47:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 11 Jan 2010 15:47:55 PST
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <17215.1263253675@parc.com>

> David Lyon <david.lyon <at> preisshare.net> writes:
> > 
> > > This has nothing to do with pushing 3.x, but all with managing
> > > available manpower and still providing quality software.
> > 
> > Python 3.x needs more carrots.

I'd be happy to move UpLib to Python 3, when the various packages that I
need are also on Python 3.  And that depresses me to think about.  I
need

   Medusa
   ReportLab
   PyLucene
   Email
   Vobject
   Mutagen
   Pyglet
   Hachoir
   Win32

I'm on the mailing lists for a lot of these, and the only one that I
know is thinking about Python 3 is Email.  I'd expect Win32 and PIL to
also be thinking about it, but I haven't heard anything.

Bill


From david.lyon at preisshare.net  Tue Jan 12 00:47:51 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:47:51 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <1220.218.214.45.58.1263253671.squirrel@syd-srv02.ezyreg.com>

Antoine writes:

> As someone who experiences the difference almost every day, I can say 3.x
> definitely has enough carrots for me. The simple fact that I will be able
> to
> raise exceptions with non-ASCII error messages without having to
> workaround a
> hopelessly broken conversion/reporting logic is a huge improvement. And
> that's
> only a small part of the picture.

In no way could I disagree with you.

Ascii works fine for us here - but I guess we're lucky.

Just keep pushing python 3 and have a nice day..

David





From barry at python.org  Tue Jan 12 01:11:28 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:11:28 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100111191128.39020d89@freewill>

On Jan 11, 2010, at 10:42 PM, Martin v. L?wis wrote:

>Wrt. the distribute feature you've noticed: Python 3.0 already supports
>that in distutils. So from day one of Python 3.x, you could have used
>that approach for porting Python code. I had been promoting it ever
>since, but it's taking up only slowly, partially because it has
>competition in the approaches "burn the bridges" (i.e. convert to 3.x
>once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
>(i.e. try to write code that works in both 2.x and 3.x unmodified).

Interesting.  The only reason I never used it before is because I didn't know
about it.  Am I alone?

Maybe I'm projecting my own preferences, but it seems to me that supporting
both Python 2 and 3 from the same code base would be a very powerful way to
enable a large amount of existing code to support Python 3.  I'm certainly
going to do this from now on with all the libraries I maintain.  I don't have
any cycles to write code twice and I can't abandon Python 2 yet.  I'm
skeptical that code can work unmodified in both 2 and 3 without 2to3.

As an example, the one library I've already ported used a metaclass.  I don't
see any way to specify that the metaclass should be used in a portable way.
In Python 2.6 it's:

class Foo:
    __metaclass__ = Meta

and in Python 3 it's:

class Foo(metaclass=Meta):

2to3 made that pain go away.

>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>2.x can do nothing:
>
>a) telling people that they have to move to 2.6 first actually
>   hurts migration, instead of helping, because it implies to them
>   that they have to drop old versions (e.g. 2.3.) - just because
>   they had *always* dropped old versions before supporting new ones.

Is it just an implication, or is it reality?  I have yet to try to support
older versions of Python and also support Python 3.  (The one package I've
done this with so far only supports Python 2.6.)

>b) IMO, people also don't gain much by first migrating to 2.6.
>   In principle, it gives them the opportunity to get py3k warnings.
>   However, I haven't heard a single positive report where these
>   warnings have actually helped people in porting. Yours is the
>   first report saying that you followed the official guideline,
>   but you didn't state whether doing so actually helped (or whether
>   you just ported to 2.6 because the guideline told you to).

Python 2.6 has other useful features, which I want to take advantage of, so
getting py3k warnings is a bonus.  Running 'python2.6 -3 setup.py test' over
my code did in fact help clean up a couple of problems.  Seems like a good
first step when considering Python 3 support to me.

>c) whatever 2.7 does (except perhaps for the warnings), it won't
>   be useful to applications, because they couldn't use it, anyway:
>   they need to support 2.4 and 2.5, and it won't have any of the
>   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
>   might actually hurt porting: people may have to special-case
>   2.7 because their work-arounds for older versions may break in 2.7
>   (e.g. testing whether a name is *not* defined, when it becomes
>   defined in 2.7), plus it gives them an incentive to not port
>   yet until they can drop support for anything before 2.7.
>
>Inherently, 2.8 can't improve on that.

I'm not so sure.  Yes, as a package maintainer there are older versions to
think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
released, what will be the minimum version of Python provided by most OS
vendors (where the majority of Python users probably get their 'python')?  I
guess some people will have to support everything from Python 2.3 to 2.8 but
you're talking supporting something like a spread of 7 years of Python
versions.  What other platform do you support for 7 years?  Even Ubuntu long
term support (LTS) releases only live for 5 years and even then, newer Pythons
are often back ported.  Or if not, then who cares?  You're not going to use a
newer version of a library on an LTS anyway.

I'm not necessarily saying that justifies a 2.8 release.  I'm just asking how
we can make it easier for people to port to Python 3.  The automatic 2to3 has
already made it easier for me to port one library to Python 3 because I barely
had to even think about it.  Maybe that's enough.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/840169eb/attachment-0005.pgp>

From barry at python.org  Tue Jan 12 01:12:19 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:12:19 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <20100111191219.5fdd2542@freewill>

On Jan 11, 2010, at 02:55 PM, Guido van Rossum wrote:

>PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
>helpful. But I trust he has ported a lot more code to 3.x than I have
>personally. Are there other experiences?

I've only done one small library so far, but it was helpful to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/ba78a57a/attachment-0005.pgp>

From andrew at bemusement.org  Tue Jan 12 01:07:26 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 12 Jan 2010 11:07:26 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100112000726.GO32351@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> I've done a fair bit of 3.x porting, and I'm firmly convinced that
> 2.x can do nothing:
[...]
> Inherently, 2.8 can't improve on that.

I agree that there are limitations like the ones you've listed, but I
disagree with your conclusion.  Maybe you assume that it's just as hard
to move to 2.8 (using the py3k backports not available in say 2.5) as it
is to 3.x?

But a hypothetical 2.8 would also give people a way to move closer to
py3k without giving up on using all their 2.x-only dependencies.  I
think it's much more likely that libraries like Twisted can support 2.8
in the near future than 3.x.

Then, when all of your dependencies (or viable alternatives to those
dependencies) are available for 3.x, you'll have an easier transition if
you can start from a 2.x with fewer differences in features.

Fundamentally the more 2.x can converge on 3.x, the easier it will be
for users to make the leap, because it will be a smaller leap.  The
longer the 2.x series lives, the more these newer 2.x versions like 2.7
and maybe even 2.8 will be available on common platforms for people to
depend upon as minimum versions, which means that as time goes by they
can depend on a version that's closer to 3.x.  And so again, the leap
becomes easier to make.  So to me it's pretty clear that 2.8 *can*
improve the transition path to 3.x.  It may or may not be a worthwhile
use of effort for python-dev to make 2.8, but that's different to saying
it's inherently pointless.

-Andrew.


From solipsis at pitrou.net  Tue Jan 12 01:28:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 00:28:26 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <loom.20100112T012550-387@post.gmane.org>

Andrew Bennetts <andrew <at> bemusement.org> writes:
> 
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies.  I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

I don't think a 2.8 could make you much closer to 3.x. AFAICT, every possible
way to ease transition to 3.x will already be in 2.7, or almost. But perhaps I'm
lacking imagination.

Regards

Antoine.




From brian.curtin at gmail.com  Tue Jan 12 02:57:46 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 11 Jan 2010 19:57:46 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>

On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:

> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
>
> -Brett
>
I think it would be interesting to see how people are using the tracker, or
how they want to be using it. For example, there are currently over 1500
open issues with no stage set, some of which seemingly haven't been read by
anyone at all. Would a properly set stage field save issues from falling
into a black hole?

Food for thought: according to the last tracker summary, there are over 1000
open issues with a patch, and issues stay open an average of 700 days
(median 450).

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/a3439436/attachment-0005.htm>

From jackdied at gmail.com  Tue Jan 12 04:53:24 2010
From: jackdied at gmail.com (Jack Diederich)
Date: Mon, 11 Jan 2010 22:53:24 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>

On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

[sidebar]
1) the metaclass fixer was a PITA to implement.
2) 95% of __metaclass__ definitions searchable via google code were of
the "__metaclass__ = type" variety.  The 2to3 patch exists only
because of the few other uses.
3) 100% of the module level assignments in public projects were the
"__metaclass__ = type" variety which is why there isn't a fixer for
that.  Also, a fixer would have been really, really ugly (munge every
class definition in this module because there is a top level
assignment).

-Jack


From benjamin at python.org  Tue Jan 12 05:01:40 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 Jan 2010 22:01:40 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <1afaf6161001112001t5e7bab83t7d93f33fa11ce849@mail.gmail.com>

2010/1/11 Jack Diederich <jackdied at gmail.com>:
> On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
>> As an example, the one library I've already ported used a metaclass. ?I don't
>> see any way to specify that the metaclass should be used in a portable way.
>> In Python 2.6 it's:
>>
>> class Foo:
>> ? ?__metaclass__ = Meta
>>
>> and in Python 3 it's:
>>
>> class Foo(metaclass=Meta):
>>
>> 2to3 made that pain go away.
>
> [sidebar]
> 1) the metaclass fixer was a PITA to implement.

Does this make it any less useful, though? :)



-- 
Regards,
Benjamin


From steven.bethard at gmail.com  Tue Jan 12 06:57:18 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 11 Jan 2010 21:57:18 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>

On Mon, Jan 11, 2010 at 4:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

Actually there's a solution to this one too:

    FooBase = Meta('FooBase', (), {})
    class Foo(FooBase):
        ...

That should work in Python 2.X and 3.X.

I've got argparse running on Python 2.3-3.1, and the changes were
pretty easy. You can see them all in the revision here:

    http://code.google.com/p/argparse/source/detail?r=12

I have aspirations of putting all of the tricks I learned up up on the
Wiki somewhere, but I just haven't had the time.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From regebro at gmail.com  Tue Jan 12 07:30:10 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:30:10 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>

On Mon, Jan 11, 2010 at 23:55, Guido van Rossum <guido at python.org> wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
>
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. But I trust he has ported a lot more code to 3.x than I have
> personally. Are there other experiences?

It doesn't warn for that many of the unportable problems, but I'm not
sure it can warn for them either. It's certainly helpful, just not
very much. :) I think the biggest help was added in 2.6.2, and that's
warning for old integer division. It will also warn for modules that
aren't supported anymore, if I remember correctly, and that's often
helpful. But it won't warn for real tricky problems, like binary vs
text in strings, and I don't see how it can either.

And, I just realized, it doesn't warn for you using cmp or __cmp__
either, and 2to3 won't fix that, so it should actually warn for it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:49:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:49:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:11, Barry Warsaw <barry at python.org> wrote:
> Interesting. ?The only reason I never used it before is because I didn't know
> about it. ?Am I alone?

Maybe not, but the Distribute feature is there because IMO the
distutils feature by itself isn't particularily useful. You need to
write your own distutils extensions, in practice, and they are not
trivial. Distribute has simply done it for you. :)

> I'm skeptical that code can work unmodified in both 2 and 3 without 2to3.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:53:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:53:20 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <319e029f1001112253x511f8109ja2300a022010ef99@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:07, Andrew Bennetts <andrew at bemusement.org> wrote:
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. ?I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

When 2.7 was discussed several people agreed that 2.7 should include
as much 3.x stuff as possible to ease transition. That turned out to
not be very much, so I'm not sure there is more. :)

Unless of course, 2.8 starts including more of the refactored
libraries, but that's a very minor issue in porting, so it won't help
much.

To really help, it needs to start implement things that break
backwards compatibility. That would be weird. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ncoghlan at gmail.com  Tue Jan 12 10:39:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:39:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4BA220.20701@voidspace.org.uk>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com> <4B4BA220.20701@voidspace.org.uk>
Message-ID: <4B4C435B.7080903@gmail.com>

Michael Foord wrote:
>> Note that the behaviour here is still different from that of a data
>> descriptor: with a data descriptor, once it gets shadowed in the
>> instance dictionary, the descriptor is ignored *completely*. The only
>> way to get the descriptor involved again is to eliminate the shadowing.
>> The non-data descriptor with only __set__ is just choosing not to
>> override the attribute lookup process.
>>
>>    
> 
> Does that mean we need a third class of descriptors that are neither
> data descriptors nor non-data descriptors?

Not really - leaving out __get__ when defining __set__ just creates a
data descriptor that happens to use the default attribute look-up
process rather than defining a different one.

(Note that I had the data/non-data terminology backwards in my previous
message - I tend to think of the two kinds of descriptor as enforced and
non-enforced respectively precisely because I have to think about it to
remember that "functions are non-data descriptors and properties are
data descriptors, hence non-data descriptors only define __get__ while
data descriptors define __set__". I don't find the data/non-data
terminology intuitive at all)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:44:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:44:18 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>	<4B4B9BB8.3070505@v.loewis.de>	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
	<319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
Message-ID: <4B4C4472.10104@gmail.com>

Lennart Regebro wrote:
> And, I just realized, it doesn't warn for you using cmp or __cmp__
> either, and 2to3 won't fix that, so it should actually warn for it.

I have a vague recollection that we tried to warn for that and ended up
nixing the warning due to vast swarms of false alarms (or because you
couldn't write correct 2.x code without triggering it).

I'd be happy for someone to prove my recollection wrong though (i.e. by
writing a patch that implements such warnings).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:48:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:48:57 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4C4589.70906@gmail.com>

David Lyon wrote:
>> This has nothing to do with pushing 3.x, but all with managing
>> available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As Guido has said a few times, the gains are far greater for *new*
Python developers than they are for existing ones.

Existing developers have to unlearn old habits and wait for libraries
they already use to get ported. New developers can just get started with
a much cleaner language. They don't have as rich a 3rd party ecosystem
on Python 3 as they would on Python 2.x at this point in time, but
unlike existing developers they also have the luxury of cherry-picking
just those packages that already have Python 3 support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Jan 12 11:20:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:20:27 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <hihida$unc$1@ger.gmane.org>

Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit?:
> 
> For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all.

I think most issues /have/ been read. It's just that for many of them, 
nobody is interested enough in or feels competent enough for fixing them.

> Would a properly set stage field save issues from
> falling into a black hole?

I don't think this has anything to do with properly setting the stage 
field. We just have limited time and manpower. Perhaps one of our goals 
should be to reach out more to potential contributors.

> Food for thought: according to the last tracker summary, there are over
> 1000 open issues with a patch, and issues stay open an average of 700
> days (median 450).

As for open issues with a patch, a "code review" button sending this 
patch to Rietveld (or any other similar tool) is something many of us 
have been hoping for :-) It would make reviewing easier, faster and more 
thorough (because you visualize things better than by looking at a diff).

Regards

Antoine.



From solipsis at pitrou.net  Tue Jan 12 11:36:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:36:49 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <hihjc1$45m$1@ger.gmane.org>

Le Tue, 12 Jan 2010 10:20:27 +0000, Antoine Pitrou a ?crit?:
> 
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.

Speaking of which, Steve had something to say about it:
http://holdenweb.blogspot.com/2009/11/comments-or-not-public-or-
private.html

cheers

Antoine.



From ncoghlan at gmail.com  Tue Jan 12 13:10:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 22:10:14 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <4B4C66A6.2040601@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
>> For example, there are currently over
>> 1500 open issues with no stage set, some of which seemingly haven't been
>> read by anyone at all.
> 
> I think most issues /have/ been read. It's just that for many of them, 
> nobody is interested enough in or feels competent enough for fixing them.

There are actually a whole host of reasons issues can stagnate:
- a feature request may seem reasonable (hence it doesn't get rejected
outright), but the right API may not be clear (hence it doesn't get
implemented in the near term)
- a patch may be reviewed and found to have significant defects (or
simply be overreaching the stated goal) but the original patch poster
loses interest after meeting resistance in their ambition to fix
something that is "obviously" broken
- a problem may simply be hard to fix in a backwards compatible way (or
even at all!)

Ultimately it boils down to Antoine's two categories (lack of interest
and lack of relevant expertise to make a final decision) but there are a
lot of different ways to get into those two buckets.

Because we aren't ruthless about pruning those kinds of issues out of
the tracker they're the ones that are going to accumulate over time.

I'd actually be interested to know what the issue stats are like when
RFEs are excluded and when the search is limited to the items flagged as
'easy'. If easy bug fix issues are taking a long time to get closed than
that would bother me a lot more than RFEs that sit on the tracker for years.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 12 13:14:47 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:14:47 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <20100112071447.675c8f24@freewill>

On Jan 11, 2010, at 10:53 PM, Jack Diederich wrote:

>3) 100% of the module level assignments in public projects were the
>"__metaclass__ = type" variety which is why there isn't a fixer for
>that.  Also, a fixer would have been really, really ugly (munge every
>class definition in this module because there is a top level
>assignment).

And almost certainly unnecessary.  IME, those are all to easily make bare
class definitions new style in Python 2.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/29b5ff24/attachment-0005.pgp>

From barry at python.org  Tue Jan 12 13:16:09 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:16:09 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
Message-ID: <20100112071609.1dcfffa6@freewill>

On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:

>Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
>That should work in Python 2.X and 3.X.

Ugly, but good call! :)

>I've got argparse running on Python 2.3-3.1, and the changes were
>pretty easy. You can see them all in the revision here:
>
>    http://code.google.com/p/argparse/source/detail?r=12
>
>I have aspirations of putting all of the tricks I learned up up on the
>Wiki somewhere, but I just haven't had the time.

The more resources we can provide people, both in code and in documentation,
the better.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/a2ba5b3e/attachment-0005.pgp>

From fuzzyman at voidspace.org.uk  Tue Jan 12 13:29:12 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 12:29:12 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112071609.1dcfffa6@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
	<20100112071609.1dcfffa6@freewill>
Message-ID: <4B4C6B18.7050008@voidspace.org.uk>

On 12/01/2010 12:16, Barry Warsaw wrote:
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>    
>> Actually there's a solution to this one too:
>>
>>     FooBase = Meta('FooBase', (), {})
>>     class Foo(FooBase):
>>         ...
>>
>> That should work in Python 2.X and 3.X.
>>      
> Ugly, but good call! :)
>
>    

There are all sorts of tricks. For example you can do exception handling 
that works with pre-2.6 syntax and 3.0 with a bare except and using 
sys.exc_info. It is horrible, but acceptable for short pieces of code (I 
have a couple of small modules that do this).

I haven't yet tried converting larger code-bases to Python 3, but I 
think the workflow advocated by Martin is greatly preferable to the 
hacks and tricks needed to make the same codebase run under 2 & 3.

Michael

>> I've got argparse running on Python 2.3-3.1, and the changes were
>> pretty easy. You can see them all in the revision here:
>>
>>     http://code.google.com/p/argparse/source/detail?r=12
>>
>> I have aspirations of putting all of the tricks I learned up up on the
>> Wiki somewhere, but I just haven't had the time.
>>      
> The more resources we can provide people, both in code and in documentation,
> the better.
>
> Thanks!
> -Barry
>    
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/7acd54f3/attachment-0005.htm>

From mal at egenix.com  Tue Jan 12 14:09:50 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 12 Jan 2010 14:09:50 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <4B4C749E.4040009@egenix.com>

Brett Cannon wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> 
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> * argparse (PEP 389)
> * brief mention on still wanting to break out the stdlib from CPython
> * an official policy on extension modules? (i.e. must have a pure Python
> implementation which can mean a ctypes implementation unless given an
> explicit waiver)
> 
> That's everything from a stdlib perspective. I have half-baked ideas, but
> nothing concrete enough to bring up unless people really want to discuss
> stuff like how to potentially standardize module deprecation warnings, etc.
> 
> In terms of me personally, I do plan to bring up at some point during the
> summit these points which don't squarely fit during my time slot:
> 
> * an official unofficial policy on how new proposed features should come to
> us (i.e. working code to python-ideas with a shell of a PEP that includes
> abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
> 
> If there is something missing from the stdlib discussion that you think
> should be brought up at the summit let me know. And if there is something
> here you want to discuss before the summit let me know and I can start a
> separate thread on it.

Could you please put these things and the results up on the Python
wiki ?!

We're going to have a language summit at EuroPython this year
as well and may want to continue/extend the discussion based
on what you're doing at PyCon.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From brian.curtin at gmail.com  Tue Jan 12 15:33:06 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 12 Jan 2010 08:33:06 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <cf9f31f21001120633n6460b55as6064228cce41c949@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:20, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
> >
> > For example, there are currently over
> > 1500 open issues with no stage set, some of which seemingly haven't been
> > read by anyone at all.
>
> I think most issues /have/ been read. It's just that for many of them,
> nobody is interested enough in or feels competent enough for fixing them.
>
> > Would a properly set stage field save issues from
> > falling into a black hole?
>
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.


Agreed, I didn't mean to place blame on the stage field, I just ran with how
I view that field since it was mentioned. When I'm thinking in "code-writing
developer mode" (tm), I'm more likely to go to issues which have a stage so
I know what I'm going into -- what needs to be worked on. When I'm in
project cleanup mode, I go by stageless issues -- what is necessary for this
to begin or end work. Maybe others work differently...software projects take
all kinds :-)

Anyways, sorry for the off-topic. If this is a summit worthy discussion,
cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/cf466340/attachment-0005.htm>

From rdmurray at bitdance.com  Tue Jan 12 17:12:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 11:12:28 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C66A6.2040601@gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org> <4B4C66A6.2040601@gmail.com>
Message-ID: <20100112161228.300EA1D688D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 22:10:14 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There are actually a whole host of reasons issues can stagnate:
> - a feature request may seem reasonable (hence it doesn't get rejected
> outright), but the right API may not be clear (hence it doesn't get
> implemented in the near term)
> - a patch may be reviewed and found to have significant defects (or
> simply be overreaching the stated goal) but the original patch poster
> loses interest after meeting resistance in their ambition to fix
> something that is "obviously" broken
> - a problem may simply be hard to fix in a backwards compatible way (or
> even at all!)

I would add:

- a patch is reviewed but the reviewer requests a second opinion before
  committing, which never arrives, and the original reviewer forgets
  about the issue.

I have occasionally observed apparently moribund issues spring to life
and get resolved when a triage person does something as simple as updating
the releases to which an issue applies.

> Ultimately it boils down to Antoine's two categories (lack of interest
> and lack of relevant expertise to make a final decision) but there are a
> lot of different ways to get into those two buckets.

There's a third bucket here: lack of time.  Sometimes there is interest
and expertise, but no time.  Worse, sometimes we end up waiting on the
person with the expertise and interest but no time, when someone else
with somewhat less expertise but more time should just go ahead and do
the commit.  (*That* one should be fixable.)

> Because we aren't ruthless about pruning those kinds of issues out of
> the tracker they're the ones that are going to accumulate over time.

Another other category that hangs around in the tracker is items for
which there simply isn't a consensus.  I suppose those could be brought
to python-dev for a final decision if someone wanted to tackle that task.

And I don't think it is a lack of ruthlessness.  I think it is the current
community consensus that we leave these issues open in the tracker in
case someone does come along and want to deal with them. (*)

> I'd actually be interested to know what the issue stats are like when
> RFEs are excluded and when the search is limited to the items flagged as
> 'easy'. If easy bug fix issues are taking a long time to get closed than
> that would bother me a lot more than RFEs that sit on the tracker for
> years.

I'd also be interested to know what the average and median lifetime
of closed bug reports is.  Not the metrics for open issues, but the
metrics for closed ones.  My theory is that we close a lot of bugs fairly
promptly, but the ones in the above categories make the average age of
*open* issues high.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls

(*) For example, I'm currently going through all the open issues
relating to the email package, and some of them are pretty darn
old, yet still have useful content.


From brett at python.org  Tue Jan 12 18:40:13 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:40:13 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C749E.4040009@egenix.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<4B4C749E.4040009@egenix.com>
Message-ID: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>

On Tue, Jan 12, 2010 at 05:09, M.-A. Lemburg <mal at egenix.com> wrote:

> Brett Cannon wrote:
> > Michael has given me the hg transition/stdlib time slot at the language
> > summit this year. In regards to that I plan to lead a discussion on:
> >
> > * where we are at w/ the Hg transition (Dirkjan should be there and I did
> a
> > blog post on this topic recently:
> > http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> > * argparse (PEP 389)
> > * brief mention on still wanting to break out the stdlib from CPython
> > * an official policy on extension modules? (i.e. must have a pure Python
> > implementation which can mean a ctypes implementation unless given an
> > explicit waiver)
> >
> > That's everything from a stdlib perspective. I have half-baked ideas, but
> > nothing concrete enough to bring up unless people really want to discuss
> > stuff like how to potentially standardize module deprecation warnings,
> etc.
> >
> > In terms of me personally, I do plan to bring up at some point during the
> > summit these points which don't squarely fit during my time slot:
> >
> > * an official unofficial policy on how new proposed features should come
> to
> > us (i.e. working code to python-ideas with a shell of a PEP that includes
> > abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> > * any changes needed to the issue tracker to help with the workflow?
> (stage
> > field seems like a failed experiment and we now have several effective
> > triage people who can help w/ guiding changes)
> >
> > If there is something missing from the stdlib discussion that you think
> > should be brought up at the summit let me know. And if there is something
> > here you want to discuss before the summit let me know and I can start a
> > separate thread on it.
>
> Could you please put these things and the results up on the Python
> wiki ?!
>
> We're going to have a language summit at EuroPython this year
> as well and may want to continue/extend the discussion based
> on what you're doing at PyCon.
>
>
I expect there will be at least summary emails on what gets discussed. There
is also a chance that it will be videotaped.

-Brett


> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 12 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/04964610/attachment-0005.htm>

From brett at python.org  Tue Jan 12 18:47:50 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:47:50 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>

On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
>
>>  * any changes needed to the issue tracker to help with the workflow?
>> (stage field seems like a failed experiment and we now have several
>> effective triage people who can help w/ guiding changes)
>>
>> -Brett
>>
> I think it would be interesting to see how people are using the tracker, or
> how they want to be using it. For example, there are currently over 1500
> open issues with no stage set, some of which seemingly haven't been read by
> anyone at all. Would a properly set stage field save issues from falling
> into a black hole?
>
>
"Using" is the key word there. I know this thread went on about how bugs
tend to end up being left open, but what I was proposing to talk about was
whether there is any shift desired by the seasoned tracker users to help in
their work. For instance, the Stage field was meant to help, but I don't
think it is really getting used much, which makes it at best just a UI junk
and at worst something to confuse new users. So I just wanted to discuss
things as a group to see if we could streamline some things, add others,
etc.

At worst I will try to grab people like Mark D., R. David, Antoine, Georg,
etc. at the summit (not sure exactly which the heavy bug closers will be
there), take them aside, and flat-out ask what they need to make their jobs
easier.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/3f808530/attachment-0005.htm>

From brett at python.org  Tue Jan 12 19:29:06 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:29:06 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C6B18.7050008@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com> 
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org> 
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill> 
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com> 
	<20100112071609.1dcfffa6@freewill> <4B4C6B18.7050008@voidspace.org.uk>
Message-ID: <bbaeab101001121029v28a863ccg8213ed494f15160c@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 12/01/2010 12:16, Barry Warsaw wrote:
>
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>
>
>  Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
> That should work in Python 2.X and 3.X.
>
>
>  Ugly, but good call! :)
>
>
>
>
> There are all sorts of tricks. For example you can do exception handling
> that works with pre-2.6 syntax and 3.0 with a bare except and using
> sys.exc_info. It is horrible, but acceptable for short pieces of code (I
> have a couple of small modules that do this).
>
> I haven't yet tried converting larger code-bases to Python 3, but I think
> the workflow advocated by Martin is greatly preferable to the hacks and
> tricks needed to make the same codebase run under 2 & 3.
>
>
In other words we need to pull together a HOWTO for Python source like the
one for extension modules that Benjamin wrote and make it rather prominently
linked from the Python 3 documentation index page.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/86cc3a21/attachment-0005.htm>

From rdmurray at bitdance.com  Tue Jan 12 19:31:23 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 13:31:23 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
Message-ID: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> >>  * any changes needed to the issue tracker to help with the workflow?
> >> (stage field seems like a failed experiment and we now have several
> >> effective triage people who can help w/ guiding changes)
> >>
> > I think it would be interesting to see how people are using the tracker, or
> > how they want to be using it. For example, there are currently over 1500
> > open issues with no stage set, some of which seemingly haven't been read by
> > anyone at all. Would a properly set stage field save issues from falling
> > into a black hole?
> >
> "Using" is the key word there. I know this thread went on about how bugs
> tend to end up being left open, but what I was proposing to talk about was
> whether there is any shift desired by the seasoned tracker users to help in
> their work. For instance, the Stage field was meant to help, but I don't
> think it is really getting used much, which makes it at best just a UI junk
> and at worst something to confuse new users. So I just wanted to discuss
> things as a group to see if we could streamline some things, add others,
> etc.

Well, I for one find the stage field useful.  It reminds me at a glance
where the issue is at in the workflow (and 'not set' is a valid place
in the workflow that an issue sometimes stays in for a while for reason
other than not having been triaged yet).  I suspect that if we discussed
it as a group (we who make the most changes on the tracker) we could
improve it, and the interface in general.

By the way, you could talk to people who aren't going to be at the
summit on #python-dev; I think all the currently tracker-active people
hang out there on a regular basis.

I'll have to give some thought to what changes/improvements might be
most useful, now that I've been doing this for a while.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Tue Jan 12 19:34:02 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:34:02 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com> 
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com> 
	<20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
Message-ID: <bbaeab101001121034m6de86385u4e0e72301e404a59@mail.gmail.com>

On Tue, Jan 12, 2010 at 10:31, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> > On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> > >>  * any changes needed to the issue tracker to help with the workflow?
> > >> (stage field seems like a failed experiment and we now have several
> > >> effective triage people who can help w/ guiding changes)
> > >>
> > > I think it would be interesting to see how people are using the
> tracker, or
> > > how they want to be using it. For example, there are currently over
> 1500
> > > open issues with no stage set, some of which seemingly haven't been
> read by
> > > anyone at all. Would a properly set stage field save issues from
> falling
> > > into a black hole?
> > >
> > "Using" is the key word there. I know this thread went on about how bugs
> > tend to end up being left open, but what I was proposing to talk about
> was
> > whether there is any shift desired by the seasoned tracker users to help
> in
> > their work. For instance, the Stage field was meant to help, but I don't
> > think it is really getting used much, which makes it at best just a UI
> junk
> > and at worst something to confuse new users. So I just wanted to discuss
> > things as a group to see if we could streamline some things, add others,
> > etc.
>
> Well, I for one find the stage field useful.  It reminds me at a glance
> where the issue is at in the workflow (and 'not set' is a valid place
> in the workflow that an issue sometimes stays in for a while for reason
> other than not having been triaged yet).  I suspect that if we discussed
> it as a group (we who make the most changes on the tracker) we could
> improve it, and the interface in general.
>
> By the way, you could talk to people who aren't going to be at the
> summit on #python-dev; I think all the currently tracker-active people
> hang out there on a regular basis.
>
>
Sounds reasonable.


> I'll have to give some thought to what changes/improvements might be
> most useful, now that I've been doing this for a while.


How about everyone who is active on bugs.python.org give it some thought and
we can try to have a discussion at some point in the relatively near future.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/26a09128/attachment-0005.htm>

From ncoghlan at gmail.com  Tue Jan 12 21:58:49 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 06:58:49 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<4B4C749E.4040009@egenix.com>
	<bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
Message-ID: <4B4CE289.6040709@gmail.com>

Brett Cannon wrote:
> I expect there will be at least summary emails on what gets discussed.
> There is also a chance that it will be videotaped.

The Wiki makes for a better summary archive though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Tue Jan 12 22:53:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:53:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
Message-ID: <4B4CEF3D.8000107@v.loewis.de>

>> a) telling people that they have to move to 2.6 first actually
>>   hurts migration, instead of helping, because it implies to them
>>   that they have to drop old versions (e.g. 2.3.) - just because
>>   they had *always* dropped old versions before supporting new ones.
> 
> Is it just an implication, or is it reality?

That's only the implication. However, this was precisely the dialogue
when talking to Django. If you start with "start supporting 2.6", the
immediate response, without listening further, was, "ok, wait until we
drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).

Then explain it to the individual you are talking to, wait for the next
developer of the project step along, and see how he brings up the
very same line of thinking (supporting new versions == dropping support
for old versions).

I think only part of that comes from the maintenance burden. The other
part is that they *want* to drop support for old versions, so that they
can eventually start using new features (e.g. generator expressions).
So they welcome the requirement to support new versions as an excuse
to drop old ones ("it is obvious that you have to drop 2.3 to support
3.2"). However, their users then won't let them drop old versions.


> 
>> b) IMO, people also don't gain much by first migrating to 2.6.
>>   In principle, it gives them the opportunity to get py3k warnings.
>>   However, I haven't heard a single positive report where these
>>   warnings have actually helped people in porting. Yours is the
>>   first report saying that you followed the official guideline,
>>   but you didn't state whether doing so actually helped (or whether
>>   you just ported to 2.6 because the guideline told you to).
> 
> Python 2.6 has other useful features, which I want to take advantage of

I think you are a minority with that, being able to actually use the 2.6
features already. Many projects can't, as they have to support at least
2.4 still (so the with statement is right out).

>> Inherently, 2.8 can't improve on that.
> 
> I'm not so sure.  Yes, as a package maintainer there are older versions to
> think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
> released, what will be the minimum version of Python provided by most OS
> vendors (where the majority of Python users probably get their 'python')?

"Current" Linux distributions will have 2.6 then. "Old" installations
will have 2.4.

> I
> guess some people will have to support everything from Python 2.3 to 2.8 but
> you're talking supporting something like a spread of 7 years of Python
> versions.  What other platform do you support for 7 years?

I think 2.3 will really be gone by the time 2.8 might get released. Even
with 2.7, you'd end up with a span of seven years, though.

Python had been supporting Windows 95 for more than 7 years (I think
rather 9 or 10), likewise Windows 3.1 before that. Python 2.7 will
likely still support Windows 2000, which then will be ten years old.

Solaris support will probably go back to Solaris 2.6, which will be
13 years old when Python 2.7 gets released.

It's only the Linux (and OS X) releases that move so quickly.

Regards,
Martin


From martin at v.loewis.de  Tue Jan 12 22:56:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:56:55 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
Message-ID: <4B4CF027.4080007@v.loewis.de>

> Maybe not, but the Distribute feature is there because IMO the
> distutils feature by itself isn't particularily useful. You need to
> write your own distutils extensions, in practice, and they are not
> trivial.

I wouldn't say that. My Django port works with bare distutils (as
does Django itself), and it works fine.

That distribute had to redo it is only because setuptools *replaces*
the build_py command, as does the 2to3 support in distutils. So only
if you have a different build_py already, you can't use what is in
distutils.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Tue Jan 12 23:02:34 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:02:34 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CEF3D.8000107@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>
	<4B4CEF3D.8000107@v.loewis.de>
Message-ID: <4B4CF17A.1000503@voidspace.org.uk>

On 12/01/2010 21:53, "Martin v. L?wis" wrote:
>>> a) telling people that they have to move to 2.6 first actually
>>>    hurts migration, instead of helping, because it implies to them
>>>    that they have to drop old versions (e.g. 2.3.) - just because
>>>    they had *always* dropped old versions before supporting new ones.
>>>        
>> Is it just an implication, or is it reality?
>>      
> That's only the implication. However, this was precisely the dialogue
> when talking to Django. If you start with "start supporting 2.6", the
> immediate response, without listening further, was, "ok, wait until we
> drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).
>
> Then explain it to the individual you are talking to, wait for the next
> developer of the project step along, and see how he brings up the
> very same line of thinking (supporting new versions == dropping support
> for old versions).
>
> I think only part of that comes from the maintenance burden. The other
> part is that they *want* to drop support for old versions, so that they
> can eventually start using new features (e.g. generator expressions).
> So they welcome the requirement to support new versions as an excuse
> to drop old ones ("it is obvious that you have to drop 2.3 to support
> 3.2"). However, their users then won't let them drop old versions.
>
>
>    

I agree with Martin that the *perception* is that to use Python 2.6 to 
help you port to Python 3 you have to be willing to drop support for 
earlier versions of Python.

>>      
>>> b) IMO, people also don't gain much by first migrating to 2.6.
>>>    In principle, it gives them the opportunity to get py3k warnings.
>>>    However, I haven't heard a single positive report where these
>>>    warnings have actually helped people in porting. Yours is the
>>>    first report saying that you followed the official guideline,
>>>    but you didn't state whether doing so actually helped (or whether
>>>    you just ported to 2.6 because the guideline told you to).
>>>        
>> Python 2.6 has other useful features, which I want to take advantage of
>>      
> I think you are a minority with that, being able to actually use the 2.6
> features already. Many projects can't, as they have to support at least
> 2.4 still (so the with statement is right out).
>
>    
Well, the IronPython community has almost completely moved over to 
IronPython 2.6. :-)

We tend to ship our Python runtime with our applications though. As it 
happens I'm now working with CPython on the server (Python 2.5) and 
IronPython in the browser where we are using 2.6. The new property 
feature is nice, as is having with without a __future__ import.

All the best,


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From regebro at gmail.com  Tue Jan 12 23:03:41 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 23:03:41 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF027.4080007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
	<4B4CF027.4080007@v.loewis.de>
Message-ID: <319e029f1001121403x14ef7ec8x729fb80fa67feaef@mail.gmail.com>

On Tue, Jan 12, 2010 at 22:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Maybe not, but the Distribute feature is there because IMO the
>> distutils feature by itself isn't particularily useful. You need to
>> write your own distutils extensions, in practice, and they are not
>> trivial.
>
> I wouldn't say that. My Django port works with bare distutils (as
> does Django itself), and it works fine.
>
> That distribute had to redo it is only because setuptools *replaces*
> the build_py command, as does the 2to3 support in distutils. So only
> if you have a different build_py already, you can't use what is in
> distutils.

Yeah, you are right, I misremembered. The actual additional feature is
the support for the test command. Testing under Python 2 and 3 without
it is annoying.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan 12 23:04:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:04:37 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <4B4CF1F5.4050403@v.loewis.de>

> [...]
>> I've done a fair bit of 3.x porting, and I'm firmly convinced that
>> 2.x can do nothing:
> [...]
>> Inherently, 2.8 can't improve on that.
> 
> I agree that there are limitations like the ones you've listed, but I
> disagree with your conclusion.  Maybe you assume that it's just as hard
> to move to 2.8 (using the py3k backports not available in say 2.5) as it
> is to 3.x?

Not at all, no. I'd rather expect that code that runs on 2.7 will run
on 2.8 unmodified.

> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. 

How so? If they use anything that is new in 2.8, they *will* need to
drop support for anything before it, no???

> I think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
that help Twisted in moving to 3.2?

> Then, when all of your dependencies (or viable alternatives to those
> dependencies) are available for 3.x, you'll have an easier transition if
> you can start from a 2.x with fewer differences in features.

But you won't *have* fewer differences. Just because your code runs
on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
for that). This doesn't get you any closer - you can't use any of
the 2.8 features as long as you have to support older versions of
Python.

> Fundamentally the more 2.x can converge on 3.x, the easier it will be
> for users to make the leap, because it will be a smaller leap.

No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
likely won't.

> The
> longer the 2.x series lives, the more these newer 2.x versions like 2.7
> and maybe even 2.8 will be available on common platforms for people to
> depend upon as minimum versions, which means that as time goes by they
> can depend on a version that's closer to 3.x.

No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
released in 2010. Then, in 2015, it may be that everybody has migrated
to 2.7, which then is a common platform.

If you release 2.8 in 2012, then, in 2015, people will be split between
2.7 and 2.8, and so there won't be a common platform before 2017.

So stopping 2.x development *earlier* will also give us a common
platform earlier.

Regareds,
Martin


From martin at v.loewis.de  Tue Jan 12 23:07:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:07:02 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <4B4CF286.5040002@v.loewis.de>

> I think it would be interesting to see how people are using the tracker,
> or how they want to be using it. For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all. Would a properly set stage field save issues from
> falling into a black hole?

I personally don't think this would be the case.

What would help is people who actually *work* on the issues, rather than
merely reading them. However, this being open source, there is no way to
force it (unless you create an incentive, such as the five-for-one
deal).

Regards,
Martin


From python at mrabarnett.plus.com  Tue Jan 12 23:10:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 12 Jan 2010 22:10:28 +0000
Subject: [Python-Dev] regex module
Message-ID: <4B4CF354.2050603@mrabarnett.plus.com>

Hi all,

I'm back on the regex module after doing other things and I'd like your
opinion on a number of matters:

Firstly, the current re module has a bug whereby it doesn't split on
zero-width matches. The BDFL has said that this behaviour should be
retained by default in case any existing software depends on it. My
question is: should my regex module still do this for Python 3?
Speaking personally, I'd like it to behave correctly, and Python 3 is
the version where backwards-compatibility is allowed to be broken.

Secondly, Python 2 is reaching the end of the line and Python 3 is the
future. Should I still release a version that works with Python 2? I'm
thinking that it could be confusing if new regex module did zero-width
splits correctly in Python 3 but not in Python 2. And also, should I
release it only for Python 3 as a 'carrot'?

Finally, the module allows some extra backslash escapes, eg \g<name>, in
the pattern. Should it treat ill-formed escapes, eg \g, as it would have
treated them in the re module?

Thanks


From michael at voidspace.org.uk  Tue Jan 12 23:46:49 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:46:49 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
Message-ID: <4B4CFBD9.1090009@voidspace.org.uk>

I presume the email below is about the Windows binary. Does the AMD64 
release work on intel 64bit and can we make the wording clearer on the 
download page?

The current description is " Windows AMD64 binary".

All the best,

Michael

-------- Original Message --------
Subject: 	Download Page - AMD64
Date: 	Fri, 11 Dec 2009 04:03:16 -0600
From: 	Thomas Brownback <thomas.brownback at gmail.com>
To: 	webmaster at python.org



Should I use the AMD64 version of Python on an Intel 64 chip? I know 
those 64-bit implementations are very similar, but are they similar 
enough that your AMD64 will work on Intel?

If so, perhaps a note should be placed on that page to help avoid 
confusion. Explicit being better than implicit and all.

Thanks for your time,
Thomas

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/80d6b539/attachment-0005.htm>

From andrew at bemusement.org  Tue Jan 12 23:49:56 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Wed, 13 Jan 2010 09:49:56 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100112224956.GC28576@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > But a hypothetical 2.8 would also give people a way to move closer to
> > py3k without giving up on using all their 2.x-only dependencies. 
> 
> How so? If they use anything that is new in 2.8, they *will* need to
> drop support for anything before it, no???
> 
> > I think it's much more likely that libraries like Twisted can support 2.8
> > in the near future than 3.x.
> 
> Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
> that help Twisted in moving to 3.2?

I'm not talking about Twisted moving to 3.x (FWIW, I think the only
movement there so far is some patches for some -3 warnings).  The
situation I'm describing is a project X that:

  (a) has 2.x-only dependencies, and
  (b) would like to be as close as possible to 3.x (because they like
      the new features and/or want to be as ready as possible to jump
      when (a) is fixed).

So just because project X depends on e.g. Twisted, and that Twisted in
turn still supports 2.4, doesn't mean that X cannot move to 2.8, and
doesn't mean it would get no benefit from doing so.

[...]
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.

But this is my point.  I think they would as an intermediate step to
jumping to 3.x (which also requires dropping 2.5, after all!), if for
some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

-Andrew.



From tjreedy at udel.edu  Tue Jan 12 23:51:31 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 17:51:31 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <hiiudf$2uv$1@ger.gmane.org>

On 1/12/2010 5:04 PM, "Martin v. L?wis" wrote:

> But you won't *have* fewer differences. Just because your code runs
> on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
> for that). This doesn't get you any closer - you can't use any of
> the 2.8 features as long as you have to support older versions of
> Python.
>
>> Fundamentally the more 2.x can converge on 3.x, the easier it will be
>> for users to make the leap, because it will be a smaller leap.
>
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.
>
>> The
>> longer the 2.x series lives, the more these newer 2.x versions like 2.7
>> and maybe even 2.8 will be available on common platforms for people to
>> depend upon as minimum versions, which means that as time goes by they
>> can depend on a version that's closer to 3.x.
>
> No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
> released in 2010. Then, in 2015, it may be that everybody has migrated
> to 2.7, which then is a common platform.
>
> If you release 2.8 in 2012, then, in 2015, people will be split between
> 2.7 and 2.8, and so there won't be a common platform before 2017.

Just like people today may need to work with both 2.5 and 2.6, or 
privately backport 2.6 bugfixes to 2.5.

> So stopping 2.x development *earlier* will also give us a common
> platform earlier.

With years of bug fixes and hence high quality.




From tjreedy at udel.edu  Wed Jan 13 00:05:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 18:05:12 -0500
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <hiiv75$5md$1@ger.gmane.org>

On 1/12/2010 5:10 PM, MRAB wrote:
> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.

Are you writing a new module with a new name? If so, do you expect it to 
replace or augment re? (This is the same question as for optparse vs. 
argparse, which I understand to not yet be decided.)
>
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?

2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
2.7 stdlib is already out. A new engine should get some community 
testing before going in the stdlib. Even 3.2 beta is not that far off 
(8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?

> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?

What does re do with analogous cases?

Terry Jan Reedy



From david.lyon at preisshare.net  Wed Jan 13 00:20:54 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 13 Jan 2010 10:20:54 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C4589.70906@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<4B4C4589.70906@gmail.com>
Message-ID: <1333.218.214.45.58.1263338454.squirrel@syd-srv02.ezyreg.com>

> Nick wrote:
>>> This has nothing to do with pushing 3.x, but all with managing
>>> available manpower and still providing quality software.
>>
>> Python 3.x needs more carrots.
>
> As Guido has said a few times, the gains are far greater for *new*
> Python developers than they are for existing ones.

Well both you and Guido are most likely 100% correct. :-)

> They don't have as rich a 3rd party ecosystem
> on Python 3 as they would on Python 2.x at this point in time, but
> unlike existing developers they also have the luxury of cherry-picking
> just those packages that already have Python 3 support.

Most likely. I wouldn't want to say anything to discourage
people from going to python 3. In other languages, I have
much experience of making the jump to 'a whole new world'.

It's unsettling at first, but after that, you suddenly
find the new model has a better turbo than the last and
you're at the next corner faster than you can think. So
it's all good.

But I still maintain Python 3.0 needs more carrots. For example,
if mercurial or any other cool lib gets added to python 3 (and
I can name a few that should go in python 3) then they should
be added to python 3 and not to python 2.x

They would serve as good carrots.

Make fresh the python 3 stdlib and preserve the python 2.x stdlib.

I really think we are somewhat short on resources to do
what Guido has asked about bringing python up to CPAN level
with respect to packages.

We're starting a long way back with horses and swords and
trying to catch a well fed and greased modern machine..

I'm sure we can get a modern package testbot operational
for python.

But I wish those who were complaining about the packaging
problem so much could throw some money at the problem
instead of moaning about it. As is done on other open-source
projects. Some organisation would be beneficial.

Finding funds so that a small group of people could
work on the problem would be a great boost to forward
progress. I just think python packaging needs six
months of that to repair the years and years of neglect
and stagnation. Even if it is only beer and bus fare
money. It just needs a temporary shot of adrenalin in
the arm.. so to speak..

Regards

David











From martin at v.loewis.de  Wed Jan 13 00:28:14 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 00:28:14 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D058E.4030404@v.loewis.de>

> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?

"intel 64bit" is as clear as mud. It could mean the "Intel 64"
architecture, or it could mean the "IA-64" architecture, both
are 64-bit architectures with processors manufactured by Intel.
The former is officially called AMD64 (not by Intel, but
officially), and our AMD64 binaries run on such processors.
The latter is also called "Itanium", and we stopped producing
binaries for that architecture a while ago; the AMD64 binaries
will *not* run on it.

The wording could probably be changed to match the reader's
expectation more, most likely, it would also get more incorrect
in the process. To make it correct and explicit, an entire
paragraph educating users about 64-bit architectures and that
there are many of them and that they are mutually incompatible
might be required.

Most likely, Thomas' processor implements the AMD64 architecture,
even though it is manufactured by Intel. A short sentence that
he would probably understand (given that he used "Intel 64", not
"64bit") is:

"""The binaries for AMD64 will also work on processors that implement
the Intel 64 architecture (formerly EM64T), i.e. the architecture that
Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
They will not work on Intel Itanium Processors (formerly IA-64)."""

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 00:30:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 00:30:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112224956.GC28576@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>	<4B4CF1F5.4050403@v.loewis.de>
	<20100112224956.GC28576@steerpike.home.puzzling.org>
Message-ID: <4B4D0613.1010503@v.loewis.de>

> I'm not talking about Twisted moving to 3.x (FWIW, I think the only
> movement there so far is some patches for some -3 warnings).  The
> situation I'm describing is a project X that:
> 
>   (a) has 2.x-only dependencies, and
>   (b) would like to be as close as possible to 3.x (because they like
>       the new features and/or want to be as ready as possible to jump
>       when (a) is fixed).

This assumes that jumping to 3.x is easier if you are closer to it.
Please trust me that this is not the case.

>> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
>> likely won't.
> 
> But this is my point.  I think they would as an intermediate step to
> jumping to 3.x (which also requires dropping 2.5, after all!), if for
> some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

No, and no. No: it's not an intermediate step, and no, supporting 3.x
does *not* require dropping 2.5.

Regards,
Martin



From fuzzyman at voidspace.org.uk  Wed Jan 13 00:40:35 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:40:35 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D058E.4030404@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
Message-ID: <4B4D0873.5070006@voidspace.org.uk>

On 12/01/2010 23:28, "Martin v. L?wis" wrote:
> [snip...]
> """The binaries for AMD64 will also work on processors that implement
> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
> They will not work on Intel Itanium Processors (formerly IA-64)."""
>
>    

To those not intimately aware of the history and present of 64 bit 
architecture this sentence would be a great addition - thanks. I'm 
adding it now as a footnote.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From lists at cheimes.de  Wed Jan 13 00:41:04 2010
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 13 Jan 2010 00:41:04 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D0890.2030505@cheimes.de>

Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64 
> release work on intel 64bit and can we make the wording clearer on the 
> download page?
> 
> The current description is " Windows AMD64 binary".

The installer works on all AMD64 compatible Intel CPUs. *scnr*

As you most likely know all modern Intel 64bit CPUs are based on AMD's
x86-64 design. Only the Itanium family is based on the other Intel 64bit
design: IA-64. The name AMD64 was chosen because most Linux and BSD
distributions call it so. The name 'AMD64' has caused confusion in the
past because users thought that the installer works only on AMD CPUs.

How about:

 * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
X86-64 binary -- does not include source)

instead of:

 * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
not include source)

?

Christia


From fuzzyman at voidspace.org.uk  Wed Jan 13 00:55:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:55:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0873.5070006@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
	<4B4D0873.5070006@voidspace.org.uk>
Message-ID: <4B4D0BD4.5050109@voidspace.org.uk>

On 12/01/2010 23:40, Michael Foord wrote:
> On 12/01/2010 23:28, "Martin v. L?wis" wrote:
>> [snip...]
>> """The binaries for AMD64 will also work on processors that implement
>> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
>> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
>> They will not work on Intel Itanium Processors (formerly IA-64)."""
>>
>
> To those not intimately aware of the history and present of 64 bit 
> architecture this sentence would be a great addition - thanks. I'm 
> adding it now as a footnote.
>

I can add it now to the main download page and existing release pages - 
but are there changes needed to any scripts to change pages created for 
*new* releases?

All the best,

Michael Foord

> All the best,
>
> Michael Foord
>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From python at mrabarnett.plus.com  Wed Jan 13 01:22:01 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 00:22:01 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <hiiv75$5md$1@ger.gmane.org>
References: <4B4CF354.2050603@mrabarnett.plus.com> <hiiv75$5md$1@ger.gmane.org>
Message-ID: <4B4D1229.70708@mrabarnett.plus.com>

Terry Reedy wrote:
> On 1/12/2010 5:10 PM, MRAB wrote:
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
> 
> Are you writing a new module with a new name? If so, do you expect it to 
> replace or augment re? (This is the same question as for optparse vs. 
> argparse, which I understand to not yet be decided.)
 >
It's a module called 'regex'. It can be used in place of 're' by using
"import regex as re", except for differences such as "\g<name>" being a
legal group reference in pattern strings.
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
> 
> 2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
> 2.7 stdlib is already out. A new engine should get some community 
> testing before going in the stdlib. Even 3.2 beta is not that far off 
> (8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?
> 
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
> 
> What does re do with analogous cases?
> 
The 're' module treats r"\g" as "g"; both 're' and 'regex' treat, say, 
r"\q" as "q". The closest analogue to what I'm asking about is that re
treats the ill-formed repeat r"x{1," as a literal, which sort of
suggests that r"\g" should be treated as "g", but r"\g<name>" is now a
group reference (re would treat that as "g<name>". Does that sound
reasonable?


From fuzzyman at voidspace.org.uk  Wed Jan 13 01:50:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 00:50:39 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D18DF.6070502@voidspace.org.uk>

On 12/01/2010 23:41, Christian Heimes wrote:
> Michael Foord wrote:
>    
>> I presume the email below is about the Windows binary. Does the AMD64
>> release work on intel 64bit and can we make the wording clearer on the
>> download page?
>>
>> The current description is " Windows AMD64 binary".
>>      
> The installer works on all AMD64 compatible Intel CPUs. *scnr*
>
> As you most likely know all modern Intel 64bit CPUs are based on AMD's
> x86-64 design. Only the Itanium family is based on the other Intel 64bit
> design: IA-64. The name AMD64 was chosen because most Linux and BSD
> distributions call it so. The name 'AMD64' has caused confusion in the
> past because users thought that the installer works only on AMD CPUs.
>
> How about:
>
>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
>
> instead of:
>
>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)
>    
Right - I've made that change for the Python 2.6, 2.7, 3.0 and 3.1 
download pages with a footnote. Prior to that we were offering ia64 
*and* amd64 releases so I've left those unchanged.

Thanks,

Michael

> ?
>
> Christia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From exarkun at twistedmatrix.com  Wed Jan 13 02:40:03 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Jan 2010 01:40:03 -0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100113014003.1898.1305834328.divmod.xquotient.219@localhost.localdomain>

On 12 Jan, 10:04 pm, martin at v.loewis.de wrote:
>>[...]
>>>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>>>2.x can do nothing:
>>[...]
>>>Inherently, 2.8 can't improve on that.
>>
>>I agree that there are limitations like the ones you've listed, but I
>>disagree with your conclusion.  Maybe you assume that it's just as 
>>hard
>>to move to 2.8 (using the py3k backports not available in say 2.5) as 
>>it
>>is to 3.x?
>
>Not at all, no. I'd rather expect that code that runs on 2.7 will run
>on 2.8 unmodified.
>>But a hypothetical 2.8 would also give people a way to move closer to
>>py3k without giving up on using all their 2.x-only dependencies.
>
>How so? If they use anything that is new in 2.8, they *will* need to
>drop support for anything before it, no???
>>I think it's much more likely that libraries like Twisted can support 
>>2.8
>>in the near future than 3.x.
>
>Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
>that help Twisted in moving to 3.2?

I'm not reading this thread carefully enough to make any arguments on 
either side, but I can contribute a fact.

Twisted very likely does not support 2.8 today.  I base this on the fact 
that Twisted does not support 2.7 today, and I expect 2.8 will be more 
like 2.7 than it will be like 2.3 - 2.6 (which Twisted does support).

When I say "support" here, I mean "all of the Twisted unit tests pass on 
it".

Jean-Paul


From tseaver at palladion.com  Wed Jan 13 03:57:46 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 12 Jan 2010 21:57:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <4B4D36AA.7020607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Karen Tracey wrote:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>>
> Well yay.  Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.  That's
> down from 104 failures and 40 errors with 2.7 alpha 1.
> 
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Just to add another "success" data point:  Zope2's trunk, as well as the
2.12 release, passes all tests (2535 on the trunk) and brings up the
appserver just fine under 2.7a2.

There is an obnoxious deprecation warning out of the distutils:

  DeprecationWarning: 'compiler' specifies the compiler type in
  build_ext. If you want to get the compiler object itself, use
  'compiler_obj'

which is likely a simple one-line fix, if I only knew what the hell it
was whining about. ;)  The warning is extra obnoxious because it doesn't
tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
argument).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktNNqQACgkQ+gerLs4ltQ7d5gCcCGKVjyOlxnrAln0UnRibS7kd
lNIAoIs1RlSGMtJWaY11BqptfDmQvR87
=mIOO
-----END PGP SIGNATURE-----



From python at mrabarnett.plus.com  Wed Jan 13 04:09:34 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 03:09:34 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <4B4D396E.4050500@mrabarnett.plus.com>

MRAB wrote:
> Hi all,
> 
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
> 
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
> 
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
> 
> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
> 
I've just noticed something odd about the re module: the sub() method
doesn't take 'pos' or 'endpos' arguments. search() does; match() does;
findall() does(); finditer() does; but sub() doesn't. Maybe there has
never been a demand for it. (Nor split(), for that matter.)


From brett at python.org  Wed Jan 13 04:58:11 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 19:58:11 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>

On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:

> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
>
>
If it is a separate module under a different name it can do the proper
thing. People will just need to be aware of the difference when they import
the module.


> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
>
>
That's totally up to you. There is practically no chance of it getting into
the 2.x under the stdlib at this point since 2.7b1 is coming up and this
module has not been out in the wild for a year (to my knowledge).  If you
want to support 2.x that's fine and I am sure users would appreciate it, but
it isn't necessary to get into the Python 3 stdlib.


> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
>
>
If you want to minimize the differences then it should probably match. As I
said, since it is a different name to import under it can deviate where
reasonable, just make sure to clearly document the deviations.

-Brett



> Thanks
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/f1f73d56/attachment-0005.htm>

From martin at v.loewis.de  Wed Jan 13 07:11:49 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 07:11:49 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D6425.3060307@v.loewis.de>

> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
> 
> instead of:
> 
>  * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)

-1. AMD doesn't want us to use the term x86-64 anymore, but wants us
to use AMD64 instead. I think we should comply - they invented the
architecture, so they have the right to give it a name. Neither
Microsoft nor Intel have such a right.

Regards,
Martin


From sridharr at activestate.com  Wed Jan 13 07:47:38 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 12 Jan 2010 22:47:38 -0800
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D6C8A.7070307@activestate.com>

On 1/12/2010 2:46 PM, Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?
>
> The current description is " Windows AMD64 binary".

FWIW, we simply use (64-bit, x64).

   Platform	          Download
   ==============================================
   Windows (x86)	          Windows Installer (MSI)
   Windows (64-bit, x64)	  Windows Installer (MSI)

https://www.activestate.com/activepython/downloads/

-srid


From solipsis at pitrou.net  Wed Jan 13 08:08:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 13 Jan 2010 07:08:55 +0000 (UTC)
Subject: [Python-Dev] Fwd: Download Page - AMD64
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <loom.20100113T080717-468@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)

+1. I don't care about trademarks or official names, we should call it whatever
is obvious for our users.
As for Itanium, it is practically dead, I think there is little chance of people
mixing it up with x86-64.

cheers

Antoine.




From michael at voidspace.org.uk  Wed Jan 13 10:32:00 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 09:32:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <4B4D9310.6060907@voidspace.org.uk>

On 13/01/2010 06:11, "Martin v. L?wis" wrote:
>> How about:
>>
>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
>>
>> instead of:
>>
>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>> not include source)
>>      
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.
>    

I think we should use whatever is most informative and least confusing 
to our users - we owe our allegiance to them and not to a processor vendor.

Michael


> Regards,
> Martin
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ncoghlan at gmail.com  Wed Jan 13 11:30:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:30:50 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF17A.1000503@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk>
Message-ID: <4B4DA0DA.7070007@gmail.com>

Michael Foord wrote:
> I agree with Martin that the *perception* is that to use Python 2.6 to
> help you port to Python 3 you have to be willing to drop support for
> earlier versions of Python.

Note that increased 3.x compatibility in the most recent 2.x release
will always help in two scenarios:

1. New projects that want to use 2.x only libraries but want to be ready
for the Py3k transition in their own code (e.g. new 2.7 features like
set literals, dict and set comprehensions and the view* dict methods can
be translated far more cleanly by 2to3 than the closest comparable 2.6
code).

2. Similarly new projects that use a 3.x trunk can be more readily
translated to a 2.7 version with 3to2, whereas some constructs may be
difficult to recreate in earlier Python versions. I would expect this to
be significantly less common then the first scenario though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:38:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:38:31 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <loom.20100113T080717-468@post.gmane.org>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<loom.20100113T080717-468@post.gmane.org>
Message-ID: <4B4DA2A7.2080408@gmail.com>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> How about:
>>
>>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
> 
> +1. I don't care about trademarks or official names, we should call it whatever
> is obvious for our users.

Until this discussion, I didn't even know the architecture had any name
other than x86-64.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:39:40 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:39:40 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4D36AA.7020607@palladion.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com>
Message-ID: <4B4DA2EC.3050908@gmail.com>

Tres Seaver wrote:
> There is an obnoxious deprecation warning out of the distutils:
> 
>   DeprecationWarning: 'compiler' specifies the compiler type in
>   build_ext. If you want to get the compiler object itself, use
>   'compiler_obj'
> 
> which is likely a simple one-line fix, if I only knew what the hell it
> was whining about. ;)  The warning is extra obnoxious because it doesn't
> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
> argument).

Could you kick a tracker issues in Tarek's direction for that one?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From vinay_sajip at yahoo.co.uk  Wed Jan 13 13:24:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 13 Jan 2010 12:24:09 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <loom.20100113T131816-515@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> If there is something missing from the stdlib discussion that you think should
be brought up at the summit let me know. And if there is something here you want
to discuss before the summit let me know and I can start a separate thread on it.

I'm sorry I won't be able to come to the language summit, but I would like if
possible to expedite a pronouncement on PEP 391 (configuring logging using
dictionaries). I believe I addressed all the comments made on the discussion
threads mentioned in the PEP and so I'm not sure what more I need to do to get a
pronouncement. I guess the stdlib slot gives an opportunity for people to air
their views and so I'd be grateful if you added it to the agenda.

Thanks & regards,

Vinay Sajip



From murman at gmail.com  Wed Jan 13 14:35:51 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 07:35:51 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>

On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.

I see and agree with the motivation behind your point, but it's just
as reasonable to mimic the name Microsoft uses: the release is for
Windows rather than the processor. On MSDN Microsoft uses
parentheticals x86, ia64 and x64; this would suggest a name like
Python 2.6.4 installer for Windows (x64). Unfortunately this usage
doesn't seem to be reflected in consumer-oriented product pages, so
I'm uncertain how clear it is for those downloading Python.

-- 
Michael Urman


From ncoghlan at gmail.com  Wed Jan 13 15:04:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 00:04:28 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
Message-ID: <4B4DD2EC.3030709@gmail.com>

Michael Urman wrote:
> On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
> 
> I see and agree with the motivation behind your point, but it's just
> as reasonable to mimic the name Microsoft uses: the release is for
> Windows rather than the processor. On MSDN Microsoft uses
> parentheticals x86, ia64 and x64; this would suggest a name like
> Python 2.6.4 installer for Windows (x64). Unfortunately this usage
> doesn't seem to be reflected in consumer-oriented product pages, so
> I'm uncertain how clear it is for those downloading Python.

As Windows doesn't run on non-x86 architectures, their naming is
generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Linux, on the other hand, can run on multiple 64 bit architectures, so
being more specific and using the official AMD64 nomenclature makes sense.

In this case, making it clear that the 64-bit Windows binaries work for
both Intel and AMD chips is more important than using only the official
architecture name.

Cheers,
Nick.

P.S. This discussion made me realise that out of habit I had dropped the
32-bit version of Python on my new 64-bit Windows box...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tseaver at palladion.com  Wed Jan 13 17:49:55 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 13 Jan 2010 11:49:55 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA2EC.3050908@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com> <4B4DA2EC.3050908@gmail.com>
Message-ID: <4B4DF9B3.4030403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> There is an obnoxious deprecation warning out of the distutils:
>>
>>   DeprecationWarning: 'compiler' specifies the compiler type in
>>   build_ext. If you want to get the compiler object itself, use
>>   'compiler_obj'
>>
>> which is likely a simple one-line fix, if I only knew what the hell it
>> was whining about. ;)  The warning is extra obnoxious because it doesn't
>> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
>> argument).
>
> Could you kick a tracker issues in Tarek's direction for that one?

Done:

  http://bugs.python.org/issue7694


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktN+bMACgkQ+gerLs4ltQ6s4gCdENhtVQWzoH449BCDz8SNx/4s
DEoAn19LMcMs3b6ctdNF8K2hYd6d+BSZ
=TWit
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Jan 13 18:24:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jan 2010 12:24:42 -0500
Subject: [Python-Dev] PYTHON3PATH
Message-ID: <20100113172442.4A7771FA679@kimball.webabinitio.net>

Please review issue 2375 [1], which is an enhancement request to add a
PYTHON3PATH environment variable.  Because we have elected to have both
a python and a python3 command, I think this is an issue worth thinking
about carefully to make sure we are serving the Python user community
and easing the transition to python3.  It could be that "use virtualenv"
is the best answer, but I feel we should think about it carefully to
make sure that is really true.

[1] http://bugs.python.org/issue2375

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From guido at python.org  Wed Jan 13 18:31:39 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:31:39 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>

Somehow the bug site doesn't load for me right now, but I'm -1 on
this. There are maybe a dozen PYTHON... variables -- we really
shouldn't try to add PYTHON3 variants for all of them.

Specifically for PYTHONPATH, I feel that its use is always a
short-time or localized hack, not something you set in your .profile
nd forget about it (forgetting about it inparticular often leads to
unnecessary debugging pain later). The better approaches are based on
site-packages, wrapper scripts, virtualenv, etc.

--Guido

On Wed, Jan 13, 2010 at 9:24 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable. ?Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3. ?It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375
>
> --
> R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From dirkjan at ochtman.nl  Wed Jan 13 18:38:26 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 13 Jan 2010 18:38:26 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <ea2499da1001130938v68827914yc41476c75b5f3c62@mail.gmail.com>

On Sat, Jan 9, 2010 at 18:29, Benjamin Peterson <benjamin at python.org> wrote:
> Please consider trying Python 2.7 with your code and reporting any bugs you may

I ran the Mercurial test suite on current trunk, and it worked
alright. There was a small issue with the fact that mimetypes got
fooled by the lack of ImportError from _winreg (which I solved by
blacklisting _winreg in demandimport), and one test fails likely
because of a change in MIME handling. Will look into that. So the
state of trunk looks rather solid from where I sit.

Cheers,

Dirkjan


From ralf at brainbot.com  Wed Jan 13 18:40:04 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:40:04 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net> (R. David
	Murray's message of "Wed, 13 Jan 2010 12:24:42 -0500")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <87r5ptdhu3.fsf@muni.brainbot.com>

"R. David Murray" <rdmurray at bitdance.com> writes:

> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable.  Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3.  It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375

The first thing I got while trying to run a python3 prompt few days ago,
was an error. python3 tried to read my $PYTHONSTARTUP file, which used
print statements. people will have to run both python 2 and python 3
code at the same time. Using different environment variables will make
this easier.

- Ralf


From ralf at brainbot.com  Wed Jan 13 18:55:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:55:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
	(Guido van Rossum's message of "Wed, 13 Jan 2010 09:31:39 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
Message-ID: <87k4vldh4c.fsf@muni.brainbot.com>

Guido van Rossum <guido at python.org> writes:

> Somehow the bug site doesn't load for me right now, but I'm -1 on
> this. There are maybe a dozen PYTHON... variables -- we really
> shouldn't try to add PYTHON3 variants for all of them.
>
> Specifically for PYTHONPATH, I feel that its use is always a
> short-time or localized hack, not something you set in your .profile
> nd forget about it (forgetting about it inparticular often leads to
> unnecessary debugging pain later). The better approaches are based on
> site-packages, wrapper scripts, virtualenv, etc.

then at least remove them completely from python3, so one can use them
for his python 2 installation. Otherwise it will stump on some innocent
python 3 program (and cause unnecessary debugging pain).



From steven.bethard at gmail.com  Wed Jan 13 18:57:29 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 13 Jan 2010 09:57:29 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:40 AM, Ralf Schmitt <ralf at brainbot.com> wrote:
> "R. David Murray" <rdmurray at bitdance.com> writes:
>
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable. ?Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3. ?It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>>
>> [1] http://bugs.python.org/issue2375
>
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

How complicated is your PYTHONSTARTUP file? My suspicion is that you
could easily write it to work for both Python 2.X and 3.X.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From ssteinerx at gmail.com  Wed Jan 13 18:57:42 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 13 Jan 2010 12:57:42 -0500
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>


On Jan 13, 2010, at 12:40 PM, Ralf Schmitt wrote:

> "R. David Murray" <rdmurray at bitdance.com> writes:
> 
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable.  Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3.  It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>> 
>> [1] http://bugs.python.org/issue2375
> 
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

Or, how about just removing the antiquated use of environment variables altogether from Python 3 and avoid the issue completely.  

Python 2 will continue to run as it has, and better solutions will emerge for Python 3 development.

Beats the heck out of making a whole duplicate set for PYTHON3BLAHBLAH, yuck!

S



From guido at python.org  Wed Jan 13 18:58:04 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:58:04 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
	<bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
Message-ID: <ca471dc21001130958k6edabaacof256b5e811c75ea6@mail.gmail.com>

Memories of days past... Python had several regular expression
implementations before, one of which was called "regex".

But I would rather not have a new module -- I would much rather have a
flag specifying the new (backwards incompatible) syntax/semantics. The
flag would have a long name (e.g. re.NEW_SYNTAX), a short name (e.g.
re.N) and an inline syntax, "(?n)...".

--Guido

On Tue, Jan 12, 2010 at 7:58 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
>>
>
> If it is a separate module under a different name it can do the proper
> thing. People will just need to be aware of the difference when they import
> the module.
>
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
>>
>
> That's totally up to you. There is practically no chance of it getting into
> the 2.x under the stdlib at this point since 2.7b1 is coming up and this
> module has not been out in the wild for a year (to my knowledge). ?If you
> want to support 2.x that's fine and I am sure users would appreciate it, but
> it isn't necessary to get into the Python 3 stdlib.
>
>>
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
>>
>
> If you want to minimize the differences then it should probably match. As I
> said, since it is a different name to import under it can deviate where
> reasonable, just make sure to clearly document the deviations.
> -Brett
>
>>
>> Thanks
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)


From ralf at brainbot.com  Wed Jan 13 19:03:08 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 19:03:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
	(Steven Bethard's message of "Wed, 13 Jan 2010 09:57:29 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
Message-ID: <87fx69dgrn.fsf@muni.brainbot.com>

Steven Bethard <steven.bethard at gmail.com> writes:

>
> How complicated is your PYTHONSTARTUP file? My suspicion is that you
> could easily write it to work for both Python 2.X and 3.X.

sure. that's exactly what I did. My point is that sharing those
environment variables will cause pain for some people.


From guido at python.org  Wed Jan 13 19:07:58 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:07:58 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
just removing the antiquated use of environment variables altogether
from Python 3 and avoid the issue completely.

-1. They have their use, but more in controlled situations. If you
have "global" env vars that you only want to use with Python 2.x,
write a wrapper for Python 3 that invokes it with -E.

-- 
--Guido van Rossum (python.org/~guido)


From scott+python-dev at scottdial.com  Wed Jan 13 19:14:21 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 13 Jan 2010 13:14:21 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E0D7D.3040806@scottdial.com>

On 1/13/2010 9:04 AM, Nick Coghlan wrote:
> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

That is not correct. There are IA-64 versions of Window Server.

>From [1]:
"Backward compatibility is a key point differentiating Itanium from x86
and x64 architectures."

So to echo what Michael said, the Microsoft nomenclature is "x64"
regardless of yours and Martin's objections to that name. Nobody who
uses Windows would be confused by "x64" since that is *the* Microsoft
naming scheme. And, anyone using IA64 will expect to see "IA64" and not
"x64", and furthermore anyone using IA64 is likely aware of what
processor they are using (being as there are only Windows Server
editions for IA64) and the difference between "IA64" and "x64".

I don't think an end-user facing page is a great place for pedantic and
wordy defenses of defying the Microsoft-blessed naming scheme.

> Linux, on the other hand, can run on multiple 64 bit architectures, so
> being more specific and using the official AMD64 nomenclature makes
> sense.

This has no relevance to the conversation since there are no Linux
binaries being distributed. The conversation on the expectations of
Windows end-users, who are the target of the download links.

[1] http://www.microsoft.com/servers/64bit/itanium/overview.mspx

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From guido at python.org  Wed Jan 13 19:22:33 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:22:33 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <loom.20100113T131816-515@post.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<loom.20100113T131816-515@post.gmane.org>
Message-ID: <ca471dc21001131022w429e4ceal899235bc711ecaca@mail.gmail.com>

On Wed, Jan 13, 2010 at 4:24 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> If there is something missing from the stdlib discussion that you think should
> be brought up at the summit let me know. And if there is something here you want
> to discuss before the summit let me know and I can start a separate thread on it.
>
> I'm sorry I won't be able to come to the language summit, but I would like if
> possible to expedite a pronouncement on PEP 391 (configuring logging using
> dictionaries). I believe I addressed all the comments made on the discussion
> threads mentioned in the PEP and so I'm not sure what more I need to do to get a
> pronouncement. I guess the stdlib slot gives an opportunity for people to air
> their views and so I'd be grateful if you added it to the agenda.

Is the PEP in the stage of consensus yet? If so it may not even be
necessary to discuss it at the summit (though I certainly won't stop
people if they want to).

-- 
--Guido van Rossum (python.org/~guido)


From phd at phd.pp.ru  Wed Jan 13 19:18:50 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 21:18:50 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <20100113181850.GA3837@phd.pp.ru>

On Wed, Jan 13, 2010 at 12:57:42PM -0500, ssteinerX at gmail.com wrote:
> Or, how about just removing the antiquated use of environment variables

   "antiquated"? You are kidding, aren't you?!

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From guido at python.org  Wed Jan 13 19:51:14 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:51:14 -0800
Subject: [Python-Dev] PyCon Keynote
Message-ID: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>

Please mail me topics you'd like to hear me talk about in my keynote
at PyCon this year.

-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Wed Jan 13 20:13:58 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:13:58 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D9310.6060907@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de> <4B4D9310.6060907@voidspace.org.uk>
Message-ID: <4B4E1B76.4010309@v.loewis.de>

>>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>> X86-64 binary -- does not include source)
>>>
>>> instead of:
>>>
>>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>> not include source)
>>>      
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
>>    
> 
> I think we should use whatever is most informative and least confusing
> to our users - we owe our allegiance to them and not to a processor vendor.

And why do you think this is x86-64?

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:33:24 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:33:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA0DA.7070007@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk> <4B4DA0DA.7070007@gmail.com>
Message-ID: <4B4E2004.8050905@v.loewis.de>

> Note that increased 3.x compatibility in the most recent 2.x release
> will always help in two scenarios:
> 
> 1. New projects that want to use 2.x only libraries but want to be ready
> for the Py3k transition in their own code (e.g. new 2.7 features like
> set literals, dict and set comprehensions and the view* dict methods can
> be translated far more cleanly by 2to3 than the closest comparable 2.6
> code).

Of these, I can only see this being the case of the view* methods.

Why would set literals allow for code that more cleanly translates
to 3.x? Suppose you write

 f = set((1,2,3))

in 2.6. That code will work *exactly* the same way in 3.1, no
translation needed at all. Likewise for dict and set comprehensions:
the code is as cleanly translated in the 2.7 form as it is in any
prior form (ie. no modifications).

As for view* methods: there is one important exception where
the 2.6 equivalent is as cleanly translated as a view method:

for key in D:
  action

This is a more modern equivalent for iterating over D.keys(),
so if your code still does the latter, just change it to the
former (requires Python 2.2). I'd claim that this is the dominant
case of traversing a dictionary (probably followed by iterating
over .items()). So while it is true that only view* can be
transformed correctly, I'd claim that this is an infrequent
issue.

> 2. Similarly new projects that use a 3.x trunk can be more readily
> translated to a 2.7 version with 3to2, whereas some constructs may be
> difficult to recreate in earlier Python versions.

That may be true, but is besides the point here: the issue was whether
a 2.8 release might help in migrating to 3.x. People who follow this
approach already have 3.x code. Whether they would rather port to 2.8
only, and get that work with little effort, or whether they would port
to 2.5 and later, and put in more work, I don't know - I guess we would
have to ask these people. I would expect that they prefer if 2.x
died rather sooner than later, because they can then stop supporting
it.

Regards,
Martin



From tjreedy at udel.edu  Wed Jan 13 20:36:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 Jan 2010 14:36:03 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <hil7av$52r$1@ger.gmane.org>

On 1/13/2010 2:13 PM, "Martin v. L?wis" wrote:

>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>
> And why do you think this is x86-64?

I do not believe I have personally seen, or at least noticed, that 
specific term before. Something like

"Release for 64-bit Windows (Win NT, 2000, XP, Vista, 7 <or whatever 
correct list is>) on AMD64-type processors from AMD and Intel"

should be clear enough.




From martin at v.loewis.de  Wed Jan 13 20:40:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 20:40:30 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E21AE.40602@v.loewis.de>

> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Windows actually does - it runs on IA-64 (which is a non-x86 architecture).

However, end users are unlikely to use such hardware, so distinguishing
between 32-bit and 64-bit is typically good enough.

> In this case, making it clear that the 64-bit Windows binaries work for
> both Intel and AMD chips is more important than using only the official
> architecture name.

Well, we did have Itanium binaries at one point, and the "64-bit Windows
binaries" you are referring to most definitely don't run on Itanium,
even though that's an Intel chip.

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:45:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:45:40 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E0D7D.3040806@scottdial.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>	<4B4DD2EC.3030709@gmail.com>
	<4B4E0D7D.3040806@scottdial.com>
Message-ID: <4B4E22E4.4040504@v.loewis.de>

> So to echo what Michael said, the Microsoft nomenclature is "x64"
> regardless of yours and Martin's objections to that name. Nobody who
> uses Windows would be confused by "x64" since that is *the* Microsoft
> naming scheme.

That's actually not entirely true. There are several places in the
APIs where Microsoft either allows or requires to call the architecture
AMD64 (e.g. architecture indication in MSI files).

Regards,
Martin


From regebro at gmail.com  Wed Jan 13 20:50:59 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 20:50:59 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>

On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

What do you need to do in the PYTHONSTARTUP file?
Ten years of Python programming, and I didn't even know it existed. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From phd at phd.pp.ru  Wed Jan 13 21:08:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 23:08:40 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <20100113200840.GC14858@phd.pp.ru>

On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From murman at gmail.com  Wed Jan 13 21:12:11 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 14:12:11 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E22E4.4040504@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com> <4B4E0D7D.3040806@scottdial.com>
	<4B4E22E4.4040504@v.loewis.de>
Message-ID: <dcbbbb411001131212q3ef907c8i67e8c03c96c31e2f@mail.gmail.com>

On Wed, Jan 13, 2010 at 13:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So to echo what Michael said, the Microsoft nomenclature is "x64"
>> regardless of yours and Martin's objections to that name. Nobody who
>> uses Windows would be confused by "x64" since that is *the* Microsoft
>> naming scheme.
>
> That's actually not entirely true. There are several places in the
> APIs where Microsoft either allows or requires to call the architecture
> AMD64 (e.g. architecture indication in MSI files).

I should have clarified I was talking about the names shown on MSDN
subscriptions for downloading installation media of Windows 7 and
Windows Vista. It's pretty clear in that context Microsoft uses x64 to
describe this platform of Windows. But again, it's far from clear that
this is a term they use for non-developers.

-- 
Michael Urman


From regebro at gmail.com  Wed Jan 13 21:27:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 21:27:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113200840.GC14858@phd.pp.ru>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<20100113200840.GC14858@phd.pp.ru>
Message-ID: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>

On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>> What do you need to do in the PYTHONSTARTUP file?
>> Ten years of Python programming, and I didn't even know it existed. :-)
>
> ? See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Cheese and fries! :-)

Anyway, I don't see how this could pose any problems, because any user
advanced enough to do these things will be advanced enough to
understand what the problem is and fix it so it works under Python 3
too.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ralf at brainbot.com  Wed Jan 13 21:52:34 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 20:52:34 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	(Lennart Regebro's message of "Wed, 13 Jan 2010 20:50:59 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <874ompg225.fsf@brainbot.com>

Lennart Regebro <regebro at gmail.com> writes:

> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
>
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

hehe. tab completion:

# -*- mode: python -*- 
# Last changed: 2009-12-23 22:25:15 by ralf

import sys
import os

def initreadline():
    
    try:
        import readline
    except ImportError:
        sys.stdout.write("Module readline not available.\n")
        return
    
    import rlcompleter
    readline.parse_and_bind("tab: complete")
    
    # Use tab for completions
    readline.parse_and_bind('tab: complete')
    # This forces readline to automatically print the above list when tab
    # completion is set to 'complete'.
    readline.parse_and_bind('set show-all-if-ambiguous on')
    # Bindings for incremental searches in the history. These searches
    # use the string typed so far on the command line and search
    # anything in the previous input history containing them.
    readline.parse_and_bind('"\C-r": reverse-search-history')
    readline.parse_and_bind('"\C-s": forward-search-history')

    history = os.path.expanduser("~/.pyhistory") 
    if os.path.exists(history):
        readline.read_history_file(history)
        
    import atexit
    atexit.register(lambda: readline.write_history_file(history))
    
initreadline()
del initreadline


From martin at v.loewis.de  Wed Jan 13 22:05:03 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 22:05:03 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <4B4E357F.4050107@v.loewis.de>

Lennart Regebro wrote:
> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
> 
> What do you need to do in the PYTHONSTARTUP file?

I personally set up readline: set tab completion, and load the history
of commands from the previous session.

Of course, I don't know what Ralf uses it for.

Regards,
Martin


From ncoghlan at gmail.com  Wed Jan 13 22:43:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 07:43:41 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
Message-ID: <4B4E3E8D.2010407@gmail.com>

Guido van Rossum wrote:
> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
> just removing the antiquated use of environment variables altogether
> from Python 3 and avoid the issue completely.
> 
> -1. They have their use, but more in controlled situations. If you
> have "global" env vars that you only want to use with Python 2.x,
> write a wrapper for Python 3 that invokes it with -E.

Perhaps a case can be made for Python 3 to assume -E by default, with a
-e option to enable reading of the environment variables?

That way naive users could run Python3 without tripping over existing
Python2 environment variables, while other tools could readily set up a
different environment before launching Python3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From guido at python.org  Wed Jan 13 23:45:57 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 14:45:57 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com> 
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com> 
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <ca471dc21001131445g639c6867ude83f03a77eb72b9@mail.gmail.com>

On Wed, Jan 13, 2010 at 1:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
>
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
>
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users using environment variables? That's a recipe for disaster
in any version! :-)

-- 
--Guido van Rossum (python.org/~guido)


From jared.grubb at gmail.com  Wed Jan 13 23:56:37 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Wed, 13 Jan 2010 14:56:37 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <13F6C43A-D62A-4A9F-AF7A-9BDAC94F7D72@gmail.com>

On 13 Jan 2010, at 13:43, Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>> 
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

I raised a question about these PYTHON3* variables once before in a discussion about shebang lines:
http://www.mailinglistarchive.com/python-dev at python.org/msg29500.html

I'm not advocating them, but just wanted to make sure to bring up the shebang use case.

Jared

From skip at pobox.com  Wed Jan 13 23:24:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 16:24:13 -0600
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <19278.18445.428716.321365@montanaro.dyndns.org>


    Lennart> What do you need to do in the PYTHONSTARTUP file?

Just reading off stuff from my own personal startup file...  I use it for
stuff I want available during interactive sessions:

    1. Enable true division.

    2. Conditionally define "help" from back in the days when there was no
       help builtin function.

    3. Auto-save session (readline) history so I can easily recall commands
       across sessions.

    4. Add other fake builtins ("like") or override behavior of some (like
       "dir") making them handier for interactive use.

    5. autoload modules/symbols (pokes around in common modules from
       sys.excepthook function).

Oh, and I've had no particular trouble keeping it working in Python 1, 2 or
3.

Skip




From fuzzyman at voidspace.org.uk  Thu Jan 14 01:20:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 00:20:21 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <4B4E6345.5070202@voidspace.org.uk>

On 13/01/2010 19:13, "Martin v. L?wis" wrote:
>>>>    * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>>> X86-64 binary -- does not include source)
>>>>
>>>> instead of:
>>>>
>>>>    * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>>> not include source)
>>>>
>>>>          
>>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>>> to use AMD64 instead. I think we should comply - they invented the
>>> architecture, so they have the right to give it a name. Neither
>>> Microsoft nor Intel have such a right.
>>>
>>>        
>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>>      
> And why do you think this is x86-64?
>    

Well anecdotal everyone I have *every* talked to about 64bit processors 
has referred to having a 64bit processor (x86 is a given) and not an 
AMD64 architecture processor.

Linus Torvalds addressed this specific issue for Linux and came down on 
the side of "x86-64": http://kerneltrap.org/node/2466
Look up AMD64 on Wikipedia and it redirects you to the X86-64 page.
Information website setup by AMD and partners about the AMD64 
architecture: http://www.x86-64.org/about.html
In the AMD website they refer to "x86-64 Assembly": 
http://www.x86-64.org/documentation/assembly.html

Microsoft seem to draw a distinction between x64 (which would also be 
acceptable) and Itanium based systems. Very rarely do they refer to AMD64:

* http://www.microsoft.com/servers/64bit/compare.mspx
* http://www.microsoft.com/servers/64bit/x64/overview.mspx
* http://www.microsoft.com/servers/64bit/overview.mspx

Using a vendor specific name automatically begs the question as to 
whether the installer works on processors from other vendors, as we saw 
in the specific enquiry from the user that triggered this debate.

Referring to the AMD 64 build as x86-64, with a footnote explaining 
which architectures this specifically means is unlikely to confuse 
people. It is *definitely* better than just saying AMD64.

All the best,

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From skip at pobox.com  Thu Jan 14 02:50:55 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 19:50:55 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
Message-ID: <19278.30847.649228.115514@montanaro.dyndns.org>


Just out of curiosity, is the static build stuff (use the old Modules/Setup
file to build modules) exercised at all any more?

Skip



From benjamin at python.org  Thu Jan 14 03:22:03 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 13 Jan 2010 20:22:03 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
In-Reply-To: <19278.30847.649228.115514@montanaro.dyndns.org>
References: <19278.30847.649228.115514@montanaro.dyndns.org>
Message-ID: <1afaf6161001131822r151bd202x35653ba44ee0235d@mail.gmail.com>

2010/1/13  <skip at pobox.com>:
>
> Just out of curiosity, is the static build stuff (use the old Modules/Setup
> file to build modules) exercised at all any more?

Exercised as in used in the build process? Yes.


-- 
Regards,
Benjamin


From vinay_sajip at yahoo.co.uk  Thu Jan 14 10:23:47 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 09:23:47 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
Message-ID: <954450.26617.qm@web25804.mail.ukl.yahoo.com>

In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:

  http://www.python.org/dev/peps/pep-0391/

In November 2009 I posted to this list that the PEP was ready for review.

I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.

So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.

Thanks & regards,

Vinay Sajip



      



From ziade.tarek at gmail.com  Thu Jan 14 10:30:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 Jan 2010 10:30:15 +0100
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <94bdd2611001140130u6154e893jfd37db32a25be66a@mail.gmail.com>

On Thu, Jan 14, 2010 at 10:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[..]
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would
> be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check
> in the changes unless there are objections to doing so. All those who think that logging is currently
> hard to configure will benefit from these changes, so here's the opportunity to pipe up and
> improve things.


FWIW, I am +1. Thanks for your work

Tarek


From hodgestar+pythondev at gmail.com  Thu Jan 14 10:39:41 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 14 Jan 2010 11:39:41 +0200
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <fb73205e1001140139n4b920871t6bfc6b91c469264f@mail.gmail.com>

On Thu, Jan 14, 2010 at 11:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I think the dict configuration scheme will be a great addition to the
logging module. :)

Schiavo
Simon


From p.f.moore at gmail.com  Thu Jan 14 12:22:09 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 Jan 2010 11:22:09 +0000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>

2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I've no immediate need for the feature, but it would be good to have
something like this, so I'm +1.
Paul.


From ncoghlan at gmail.com  Thu Jan 14 12:46:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:46:19 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
Message-ID: <4B4F040B.8010607@gmail.com>

Paul Moore wrote:
> 2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> So, can you please indicate your vote for or against incorporating PEP 391 into Python?
> 
> I've no immediate need for the feature, but it would be good to have
> something like this, so I'm +1.

I'm in the same boat as Paul, but PEP 291 provides a nice forward
compatible configuration scheme that will work with any application
configuration approach that can produce an appropriate Python dictionary.

So +1 from me too - I think the PEP has now taken this as far as
theorising can go, and the only way to learn anything further is to put
it into practice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Jan 14 12:57:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:57:55 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4B4F06C3.7030806@gmail.com>

Vinay Sajip wrote:
> In October 2009 I created PEP 391 to propose a new method of
> configuring logging using dictionaries:
> 
> http://www.python.org/dev/peps/pep-0391/

Although one minor comment: you can probably remove the note about the
"ext://" and "cfg://" prefixes being provisional at this stage. Those
names look fine to me, so I don't see any point inviting a late-breaking
bikeshed discussion about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Thu Jan 14 14:53:29 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 08:53:29 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>

On Thu, Jan 14, 2010 at 4:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:
>
> ?http://www.python.org/dev/peps/pep-0391/
>
> In November 2009 I posted to this list that the PEP was ready for review.
>
> I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.
>
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.
>
> Thanks & regards,
>
> Vinay Sajip

I'm generally +1 - but given I know that Django 1.2 is slated to
implement something somewhat similar, I'm interested to hear how this
proposal meshes with their plan(s).

jesse


From vinay_sajip at yahoo.co.uk  Thu Jan 14 15:08:24 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 14:08:24 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
Message-ID: <92210.91656.qm@web25806.mail.ukl.yahoo.com>

> From: Jesse Noller <jnoller at gmail.com>

> I'm generally +1 - but given I know that Django 1.2 is slated to
> implement something somewhat similar, I'm interested to hear how this
> proposal meshes with their plan(s)..

Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:

http://groups.google.com/group/django-developers/msg/4ef81a2160257221

They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).

Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

Regards,

Vinay Sajip


      



From ssteinerx at gmail.com  Thu Jan 14 15:54:27 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 14 Jan 2010 09:54:27 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <62E362ED-5DD7-4D42-B5E0-B263A137FD5C@gmail.com>


On Jan 14, 2010, at 9:08 AM, Vinay Sajip wrote:

>> From: Jesse Noller <jnoller at gmail.com>
> 
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
> 
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
> 
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
> 
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
> 
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

That puts a huge +1 on there for me.  

If we can get this approved and have Django's logging improved *and* avoid a whole pile of duplicate effort in Django that's a huge win.

S



From jnoller at gmail.com  Thu Jan 14 15:56:18 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 09:56:18 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140656w68c7290agccfe7282cf96f2fb@mail.gmail.com>

On Thu, Jan 14, 2010 at 9:08 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> From: Jesse Noller <jnoller at gmail.com>
>
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
>
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
>
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
>
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
>
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.
>
> Regards,
>
> Vinay Sajip
>

Cool, +1 then :)


From mal at egenix.com  Thu Jan 14 20:19:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 20:19:12 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <4B4F6E30.50400@egenix.com>

Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users won't have PYTHONPATH or any other Python environment
variables setup.

Really, if you know that you are going to run Python 3 instead of
Python 2 or vice-versa it's easy enough to run

. py3env.sh
or
. py2env.sh

in order to setup your shell environment, much like you typically
do when using virtual Python installations or work on different
projects that require different setups.

If you just want to separate Python 2 and 3 files, use the
user site-packages dir which includes the Python version.

More experienced users could also write their own environment
switching sitecustomize.py or usercustomize.py.

And then there's the hackery option for experts that love
dirty tricks: add a .pth file which includes an "import mysetup"
line to fix-up you path and other settings.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Jan 14 22:02:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 15 Jan 2010 07:02:28 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <4B4F8664.4080107@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Guido van Rossum wrote:
>>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>>> just removing the antiquated use of environment variables altogether
>>> from Python 3 and avoid the issue completely.
>>>
>>> -1. They have their use, but more in controlled situations. If you
>>> have "global" env vars that you only want to use with Python 2.x,
>>> write a wrapper for Python 3 that invokes it with -E.
>> Perhaps a case can be made for Python 3 to assume -E by default, with a
>> -e option to enable reading of the environment variables?
>>
>> That way naive users could run Python3 without tripping over existing
>> Python2 environment variables, while other tools could readily set up a
>> different environment before launching Python3.
> 
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.

I was actually thinking of the situation where the OS had a preinstalled
Python 2 with a default PYTHONPATH setting and the user was playing with
Python 3.

However, I agree that that is a fairly unlikely scenario (since
preinstalled Pythons tend not to rely on the environment variables and a
user sophisticated enough to be playing with a new Python interpreter
should be able to cope with a few environment variable conflicts anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at voidspace.org.uk  Thu Jan 14 22:09:30 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 21:09:30 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F8664.4080107@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <4B4F8664.4080107@gmail.com>
Message-ID: <4B4F880A.5010809@voidspace.org.uk>

On 14/01/2010 21:02, Nick Coghlan wrote:
> However, I agree that that is a fairly unlikely scenario (since
> preinstalled Pythons tend not to rely on the e
Well, on the other hand I think that during the next few years it will 
be increasingly common for developers (and possibly users) to have 
Python 2 and Python 3 installed side-by-side.

Many libraries and applications may never make the jump to Python 3 and 
Python users may be using 'legacy' Python 2 code for many years to come. 
It will also become increasingly common for developers to be using 
Python 3 *primarily* and for Python 3 only libraries and applications to 
emerge.

Whilst there are workarounds we *are* in a situation that Python 2 and 
Python 3 share environment variables for the location of libraries and 
executing code on startup, whilst at the same time they are largely 
incompatible and need separate library paths and startup code.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ralf at brainbot.com  Thu Jan 14 22:25:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 14 Jan 2010 22:25:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com> (M.'s message of "Thu, 14 Jan 2010
	20:19:12 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <871vhswf90.fsf@brainbot.com>

"M.-A. Lemburg" <mal at egenix.com> writes:

>
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.
>
> Really, if you know that you are going to run Python 3 instead of
> Python 2 or vice-versa it's easy enough to run

You don't even know that you're going to run python. I have 40 python
scripts in my /usr/bin directory. 

>
> . py3env.sh
> or
> . py2env.sh
>
> in order to setup your shell environment, much like you typically
> do when using virtual Python installations or work on different
> projects that require different setups.

No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.

>
> If you just want to separate Python 2 and 3 files, use the
> user site-packages dir which includes the Python version.

Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
install those programs in a user's home directory. There are still
people running python <2.6.


From mal at egenix.com  Thu Jan 14 22:51:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 22:51:07 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <871vhswf90.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <871vhswf90.fsf@brainbot.com>
Message-ID: <4B4F91CB.2060106@egenix.com>

Ralf Schmitt wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>
>> Naive users won't have PYTHONPATH or any other Python environment
>> variables setup.
>>
>> Really, if you know that you are going to run Python 3 instead of
>> Python 2 or vice-versa it's easy enough to run
> 
> You don't even know that you're going to run python. I have 40 python
> scripts in my /usr/bin directory. 
> 
>>
>> . py3env.sh
>> or
>> . py2env.sh
>>
>> in order to setup your shell environment, much like you typically
>> do when using virtual Python installations or work on different
>> projects that require different setups.
> 
> No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.
> 
>>
>> If you just want to separate Python 2 and 3 files, use the
>> user site-packages dir which includes the Python version.
> 
> Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
> install those programs in a user's home directory. There are still
> people running python <2.6.

Note that you only need to have a different PYTHONPATH
setups for Python 2.x/3.x if you plan to run code that:

 * is not installed in site-packages (most OS shipped code
   will be found in the resp. system site-packages/ dir)

 * is not installed in a user site-packages directory
   (user installed code will typically go there (*))

 * uses modules/packages that come in two different versions
   (one for Python 2.x and one for 3.x) and use the same name
   for both versions

 * needs to work in both Python 2.x and 3.x


(*) The method for installing code in user site-packages dir is
running:

    python setup.py install --user

instead of the standard

    python setup.py install

which install to the system-wide site-packages.

BTW: I guess the bzr and mercurial wikis will need to be updated
accordingly - at least for users of Python >=2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From martin at v.loewis.de  Thu Jan 14 22:55:04 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Jan 2010 22:55:04 +0100
Subject: [Python-Dev] [ANN] Python 2.5.5 Release Candidate 1.
Message-ID: <4B4F92B8.30806@v.loewis.de>

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.5.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.6 (which is
2.6.4 at this point).

This releases fixes issues with the logging and tarfile modules, and
with thread-local variables. See the detailed release notes at the
website (also available as Misc/NEWS in the source distribution) for
details of bugs fixed.

For more information on Python 2.5.5, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.5.5

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

    http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
martin at v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)


From status at bugs.python.org  Fri Jan 15 18:07:26 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jan 2010 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100115170726.9963A7865F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/08/10 - 01/15/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2536 open (+32) / 16993 closed (+16) / 19529 total (+48)

Open issues with patches:  1024

Average duration of open issues: 710 days.
Median duration of open issues: 469 days.

Open Issues Breakdown
   open  2501 (+32)
pending    34 ( +0)

Issues Created Or Reopened (53)
_______________________________

PYTHON3PATH environment variable to supersede PYTHONPATH for mul 01/13/10
CLOSED http://bugs.python.org/issue2375    reopened r.david.murray                
                                                                               

Mark the compiler package as deprecated                          01/13/10
       http://bugs.python.org/issue6837    reopened ezio.melotti                  
                                                                               

test_distutils failure                                           01/15/10
CLOSED http://bugs.python.org/issue6961    reopened flox                          
       buildbot                                                                

test_urllib: unsetting missing 'env' variable                    01/08/10
CLOSED http://bugs.python.org/issue7026    reopened flox                          
                                                                               

Two float('nan') are not equal                                   01/08/10
CLOSED http://bugs.python.org/issue7660    created  jmfauth                       
                                                                               

compiling ctypes fails with non-ascii path                       01/08/10
       http://bugs.python.org/issue7661    created  pitrou                        
       patch                                                                   

time.utcoffset()                                                 01/09/10
       http://bugs.python.org/issue7662    created  techtonik                     
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points  01/10/10
CLOSED http://bugs.python.org/issue7663    created  exarkun                       
                                                                               

python logger does not handle IOError Exception                  01/10/10
CLOSED http://bugs.python.org/issue7664    created  yateenjoshi                   
                                                                               

test_urllib2 and test_ntpath fail if path contains "\"           01/10/10
       http://bugs.python.org/issue7665    created  flox                          
                                                                               

test_lib2to3 fails if path contains space                        01/10/10
CLOSED http://bugs.python.org/issue7666    created  flox                          
                                                                               

test_doctest fails with non-ascii path                           01/10/10
       http://bugs.python.org/issue7667    created  flox                          
       buildbot                                                                

test_httpservers fails with non-ascii path                       01/10/10
       http://bugs.python.org/issue7668    created  flox                          
       buildbot                                                                

test_unicode_file fails with non-ascii path                      01/10/10
CLOSED http://bugs.python.org/issue7669    created  flox                          
                                                                               

_sqlite3: Block *all* operations on a closed Connection object   01/10/10
       http://bugs.python.org/issue7670    created  haypo                         
       patch                                                                   

test_popen fails if path contains special char like ";"          01/11/10
       http://bugs.python.org/issue7671    reopened flox                          
       patch                                                                   

_ssl module causes segfault                                      01/10/10
       http://bugs.python.org/issue7672    created  ssoria                        
       patch                                                                   

audioop: check that length is a multiple of the size             01/11/10
       http://bugs.python.org/issue7673    created  haypo                         
       patch                                                                   

select.select() corner cases: duplicate fds, out-of-range fds    01/11/10
       http://bugs.python.org/issue7674    created  cdleary                       
                                                                               

help() doesn't accept unicode literals in built in docstrings    01/11/10
       http://bugs.python.org/issue7675    created  psd                           
       patch                                                                   

IDLE shell shouldn't use TABs                                    01/11/10
       http://bugs.python.org/issue7676    created  cben                          
                                                                               

distutils, better error message for setup.py upload -sign withou 01/11/10
       http://bugs.python.org/issue7677    created  illume                        
                                                                               

subprocess.Popen pipeline example code in the documentation is l 01/11/10
       http://bugs.python.org/issue7678    created  steven.k.wong                 
                                                                               

Warning building 2.7 on OS X 10.6 libintl.h "Present But Cannot  01/12/10
       http://bugs.python.org/issue7679    created  ssteinerX                     
                                                                               

pythonw crash while attempting to start() a thread object        01/12/10
       http://bugs.python.org/issue7680    created  dontbugme                     
                                                                               

Incorrect division in [wave.py]                                  01/12/10
CLOSED http://bugs.python.org/issue7681    created  alfps                         
       patch, needs review                                                     

Optimisation of if with constant expression                      01/12/10
       http://bugs.python.org/issue7682    created  sdefresne                     
       patch                                                                   

Wrong link in HTML documentation                                 01/12/10
CLOSED http://bugs.python.org/issue7683    created  francescor                    
                                                                               

decimal.py: infinity coefficients in tuples                      01/12/10
       http://bugs.python.org/issue7684    created  skrah                         
                                                                               

minor typo in re docs                                            01/12/10
CLOSED http://bugs.python.org/issue7685    created  mikejs                        
       patch                                                                   

redundant open modes 'rbb', 'wbb', 'abb' no longer work on Windo 01/13/10
       http://bugs.python.org/issue7686    created  ivank                         
                                                                               

Bluetooth support untested                                       01/13/10
       http://bugs.python.org/issue7687    created  pitrou                        
                                                                               

TypeError: __name__ must be set to a string object               01/13/10
       http://bugs.python.org/issue7688    created  frankmillman                  
                                                                               

Pickling of classes with a metaclass and copy_reg                01/13/10
       http://bugs.python.org/issue7689    created  craigcitro                    
       patch                                                                   

Wrong PEP number in importlib module manual page                 01/13/10
CLOSED http://bugs.python.org/issue7690    created  francescor                    
                                                                               

test_bsddb3 files are not always removed when test fails         01/13/10
CLOSED http://bugs.python.org/issue7691    created  flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero              01/13/10
CLOSED http://bugs.python.org/issue7692    created  Bugger                        
                                                                               

tarfile.extractall can't have unicode extraction path            01/13/10
       http://bugs.python.org/issue7693    created  pbienst                       
                                                                               

DeprecationWarning from distuils.commands.build_ext needs stackl 01/13/10
       http://bugs.python.org/issue7694    created  tseaver                       
       patch                                                                   

missing termios constants                                        01/13/10
       http://bugs.python.org/issue7695    created  conorh                        
                                                                               

Improve Memoryview/Buffer documentation                          01/13/10
       http://bugs.python.org/issue7696    created  flox                          
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes  01/13/10
CLOSED http://bugs.python.org/issue7697    created  flox                          
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame  01/14/10
CLOSED http://bugs.python.org/issue7698    created  exarkun                       
       patch                                                                   

strptime, strftime documentation                                 01/14/10
       http://bugs.python.org/issue7699    created  mikejs                        
       patch                                                                   

"-3" flag does not work anymore                                  01/14/10
CLOSED http://bugs.python.org/issue7700    created  flox                          
       patch                                                                   

fix output string length for binascii.b2a_uu()                   01/14/10
CLOSED http://bugs.python.org/issue7701    created  haypo                         
       patch                                                                   

Wrong order of parameters of _get_socket in SMTP class in smtpli 01/14/10
       http://bugs.python.org/issue7702    created  alito                         
                                                                               

Replace buffer()-->memoryview() in Lib/ctypes/test/              01/14/10
       http://bugs.python.org/issue7703    created  flox                          
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE         01/14/10
CLOSED http://bugs.python.org/issue7704    created  DhaReaL                       
                                                                               

libpython2.6.so is not linked correctly on FreeBSD when threads  01/15/10
       http://bugs.python.org/issue7705    created  aballier                      
       patch, needs review                                                     

Missing #include guards                                          01/15/10
       http://bugs.python.org/issue7706    created  akrpic77                      
       patch                                                                   

multiprocess.Queue operations during import can lead to deadlock 01/15/10
       http://bugs.python.org/issue7707    created  kripken                       
                                                                               

Conversion of longs to bytes and vice-versa.                     01/11/10
       http://bugs.python.org/issue1023290 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (67)
______________________

segfault in curses when calling redrawwin() before refresh()      825 days
       http://bugs.python.org/issue1266    mark.dickinson                
                                                                               

"RuntimeError: dictionary changed size during iteration" in Tkin  751 days
       http://bugs.python.org/issue1658    flox                          
       patch                                                                   

Exception exceptions.AttributeError '_shutdown' in <module 'thre  718 days
       http://bugs.python.org/issue1947    nadiasvertex                  
                                                                               

Backport dictviews to 2.7                                         713 days
       http://bugs.python.org/issue1967    alexandre.vassalotti          
       patch, needs review                                                     

Backport set and dict comprehensions                              665 days
       http://bugs.python.org/issue2333    alexandre.vassalotti          
       patch, 26backport                                                       

Backport set literals                                             663 days
       http://bugs.python.org/issue2335    alexandre.vassalotti          
       patch, 26backport                                                       

PYTHON3PATH environment variable to supersede PYTHONPATH for mul    1 days
       http://bugs.python.org/issue2375    lemburg                       
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     625 days
       http://bugs.python.org/issue2698    cmcqueen1975                  
                                                                               

arg 2 of PyErr_SetFromErrnoWithFilename should be const           619 days
       http://bugs.python.org/issue2758    brian.curtin                  
                                                                               

Trunk build issues in DragonFly BSD                               613 days
       http://bugs.python.org/issue2797    brian.curtin                  
       patch, needs review                                                     

Gzip cannot handle zero-padded output + patch                     610 days
       http://bugs.python.org/issue2846    pitrou                        
       patch, needs review                                                     

block operation on closed socket/pipe for multiprocessing         552 days
       http://bugs.python.org/issue3311    haypo                         
       patch, needs review                                                     

Add gamma function, error functions and other C99 math.h functio  543 days
       http://bugs.python.org/issue3366    mark.dickinson                
       patch, needs review                                                     

Macros for PyLong: sign, number of digits, fits in an int         425 days
       http://bugs.python.org/issue4294    mark.dickinson                
       patch                                                                   

reject unicode in zlib                                            379 days
       http://bugs.python.org/issue4757    haypo                         
       patch                                                                   

Distutils inappropriately reuses .o files between extension modu  320 days
       http://bugs.python.org/issue5372    tarek                         
       patch, patch, needs review                                              

Problem with email.MIME* library, using wrong new line            298 days
       http://bugs.python.org/issue5525    r.david.murray                
                                                                               

os.path.normpath doesn't preserve unicode                         263 days
       http://bugs.python.org/issue5827    ezio.melotti                  
       patch                                                                   

smtplib exception smtp.connect TypeError encode_plain             173 days
       http://bugs.python.org/issue6523    r.david.murray                
                                                                               

email.parser clips trailing \n of multipart/mixed part if part e  152 days
       http://bugs.python.org/issue6681    r.david.murray                
       patch                                                                   

Optimize PyBytes_FromObject.                                      151 days
       http://bugs.python.org/issue6688    alexandre.vassalotti          
       patch                                                                   

in xmlrpclib.py: NameError: global name 'HTTPSConnection' is not  143 days
       http://bugs.python.org/issue6769    krisvale                      
                                                                               

UnicodeDecodeError when retrieving binary data from cgi.FieldSto  126 days
       http://bugs.python.org/issue6854    r.david.murray                
                                                                               

test_distutils failure                                              0 days
       http://bugs.python.org/issue6961    flox                          
       buildbot                                                                

tmpnam should not be used if tempfile or mkstemp are available    110 days
       http://bugs.python.org/issue6965    flox                          
                                                                               

test_urllib: unsetting missing 'env' variable                       0 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

distutils and IronPython compatibility                             73 days
       http://bugs.python.org/issue7071    tarek                         
       26backport                                                              

weak dict iterators are fragile because of unpredictable GC runs   89 days
       http://bugs.python.org/issue7105    pitrou                        
       patch                                                                   

email:  msg.items() returns different values before and after ms   89 days
       http://bugs.python.org/issue7119    r.david.murray                
       patch                                                                   

get_payload(decode=True) eats last newline                         87 days
       http://bugs.python.org/issue7143    r.david.murray                
                                                                               

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo   49 days
       http://bugs.python.org/issue7382    alexandre.vassalotti          
       patch, easy                                                             

Document inspect.get(full)argspec limitation to Python function    39 days
       http://bugs.python.org/issue7422    georg.brandl                  
                                                                               

Py3.1: Fatal Python Error: Py_Initialize...unknown encoding: chc   35 days
       http://bugs.python.org/issue7441    georg.brandl                  
                                                                               

when piping output between subprocesses some fd is left open blo   36 days
       http://bugs.python.org/issue7448    flox                          
                                                                               

Solaris SPARC: _multiprocessing.so: symbol sem_timedwait: refere   35 days
       http://bugs.python.org/issue7454    srid                          
                                                                               

cPickle: stack underflow in load_pop()                             33 days
       http://bugs.python.org/issue7455    haypo                         
       patch                                                                   

crash in str.rfind() with an invalid start value                   33 days
       http://bugs.python.org/issue7458    haypo                         
       patch                                                                   

test_multiprocessing test_rapid_restart fails if port 9999 alrea   27 days
       http://bugs.python.org/issue7498    r.david.murray                
       patch, buildbot                                                         

Extended slicing with classic class behaves strangely               3 days
       http://bugs.python.org/issue7532    mark.dickinson                
       patch                                                                   

stringlib fastsearch could be improved on 64-bit builds            13 days
       http://bugs.python.org/issue7607    pitrou                        
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option    7 days
       http://bugs.python.org/issue7617    tarek                         
       patch                                                                   

[patch] improve unicode methods: split() rsplit() and replace()    10 days
       http://bugs.python.org/issue7622    pitrou                        
       patch                                                                   

bytearray needs more tests for "b.some_method()[0] is not b"       10 days
       http://bugs.python.org/issue7625    pitrou                        
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                    4 days
       http://bugs.python.org/issue7648    orsenthil                     
       easy                                                                    

[patch] Enable additional bytes and memoryview tests.               5 days
       http://bugs.python.org/issue7654    pitrou                        
       patch                                                                   

test_ctypes failure on AIX 5.3                                      4 days
       http://bugs.python.org/issue7657    mallayya                      
       patch                                                                   

Two float('nan') are not equal                                      0 days
       http://bugs.python.org/issue7660    mark.dickinson                
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points     1 days
       http://bugs.python.org/issue7663    lemburg                       
                                                                               

python logger does not handle IOError Exception                     1 days
       http://bugs.python.org/issue7664    vinay.sajip                   
                                                                               

test_lib2to3 fails if path contains space                           0 days
       http://bugs.python.org/issue7666    benjamin.peterson             
                                                                               

test_unicode_file fails with non-ascii path                         2 days
       http://bugs.python.org/issue7669    ezio.melotti                  
                                                                               

Incorrect division in [wave.py]                                     1 days
       http://bugs.python.org/issue7681    benjamin.peterson             
       patch, needs review                                                     

Wrong link in HTML documentation                                    0 days
       http://bugs.python.org/issue7683    ezio.melotti                  
                                                                               

minor typo in re docs                                               0 days
       http://bugs.python.org/issue7685    ezio.melotti                  
       patch                                                                   

Wrong PEP number in importlib module manual page                    0 days
       http://bugs.python.org/issue7690    brett.cannon                  
                                                                               

test_bsddb3 files are not always removed when test fails            2 days
       http://bugs.python.org/issue7691    flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero                 0 days
       http://bugs.python.org/issue7692    mark.dickinson                
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes     0 days
       http://bugs.python.org/issue7697    pjenvey                       
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame     0 days
       http://bugs.python.org/issue7698    skip.montanaro                
       patch                                                                   

"-3" flag does not work anymore                                     1 days
       http://bugs.python.org/issue7700    brett.cannon                  
       patch                                                                   

fix output string length for binascii.b2a_uu()                      1 days
       http://bugs.python.org/issue7701    pitrou                        
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE            0 days
       http://bugs.python.org/issue7704    tim_one                       
                                                                               

Support for sending multipart form data                          2452 days
       http://bugs.python.org/issue727898  r.david.murray                
                                                                               

email.MIME*.as_string removes duplicate spaces                   1440 days
       http://bugs.python.org/issue1422094 r.david.murray                
       easy                                                                    

test_parsedate_acceptable_to_time_functions+DST == :-(           1394 days
       http://bugs.python.org/issue1454285 r.david.murray                
                                                                               

email module does not complay with RFC 2046: CRLF issue          1196 days
       http://bugs.python.org/issue1571841 r.david.murray                
                                                                               

email.FeedParser.BufferedSubFile improperly handles "\r\n"        969 days
       http://bugs.python.org/issue1721862 r.david.murray                
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 20 dtoa.c: oversize b in quorem                                      11 days
open    http://bugs.python.org/issue7632   

 18 _ssl module causes segfault                                        5 days
open    http://bugs.python.org/issue7672   

 12 Speed up cPickle's pickling generally                            287 days
open    http://bugs.python.org/issue5683   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    7 days
open    http://bugs.python.org/issue7658   

  8 Fatal error on thread creation in low memory condition            27 days
open    http://bugs.python.org/issue7544   

  8 Direct calls to PyObject_Del/PyObject_DEL are broken for --with  558 days
open    http://bugs.python.org/issue3299   

  7 "-3" flag does not work anymore                                    1 days
closed  http://bugs.python.org/issue7700   

  7 tarfile.extractall can't have unicode extraction path              2 days
open    http://bugs.python.org/issue7693   

  7 test_urllib: unsetting missing 'env' variable                      0 days
closed  http://bugs.python.org/issue7026   

  7 os.path.abspath with unicode argument should call os.getcwdu     542 days
open    http://bugs.python.org/issue3426   




From g.brandl at gmx.net  Sat Jan 16 21:05:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 16 Jan 2010 21:05:46 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<20100113200840.GC14858@phd.pp.ru>
	<319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
Message-ID: <hit67o$7v4$1@ger.gmane.org>

Am 13.01.2010 21:27, schrieb Lennart Regebro:
> On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
>> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>>> What do you need to do in the PYTHONSTARTUP file?
>>> Ten years of Python programming, and I didn't even know it existed. :-)
>>
>>   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.
> 
> Cheese and fries! :-)
> 
> Anyway, I don't see how this could pose any problems, because any user
> advanced enough to do these things will be advanced enough to
> understand what the problem is and fix it so it works under Python 3
> too.

I'd propose adding a bit of text to the environment variables documentation,
and especially about the needs of PYTHONSTARTUP files.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From asmodai at in-nomine.org  Sat Jan 16 21:50:56 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 16 Jan 2010 21:50:56 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <874ompg225.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
Message-ID: <20100116205056.GL99670@nexus.in-nomine.org>

-On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>hehe. tab completion:

With bpython and ipython available, why would you even want to stick to the
'plain old' interactive interpreter?

(Sorry to derail the discussion, but maybe there's more people that have not
heard of either or both.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For ever, brother, hail and farewell...


From solipsis at pitrou.net  Sat Jan 16 21:57:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jan 2010 20:57:13 +0000 (UTC)
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <loom.20100116T215639-769@post.gmane.org>

Jeroen Ruigrok van der Werven <asmodai <at> in-nomine.org> writes:
> 
> -On [20100113 22:13], Ralf Schmitt (ralf <at> brainbot.com) wrote:
> >hehe. tab completion:
> 
> With bpython and ipython available, why would you even want to stick to the
> 'plain old' interactive interpreter?

Why wouldn't we?
There are probably many more people using the standard Python prompt than
ipython.





From ben+python at benfinney.id.au  Sat Jan 16 23:41:03 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2010 09:41:03 +1100
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <87y6jxk70g.fsf@benfinney.id.au>

Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:

> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
> >hehe. tab completion:
>
> With bpython and ipython available, why would you even want to stick
> to the 'plain old' interactive interpreter?

Because those optional extras are not always available, whereas the
standard interactive interpreter is always available with CPython
distributions.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Sun Jan 17 01:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:22:10 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87y6jxk70g.fsf@benfinney.id.au>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<874ompg225.fsf@brainbot.com>	<20100116205056.GL99670@nexus.in-nomine.org>
	<87y6jxk70g.fsf@benfinney.id.au>
Message-ID: <4B525832.8090904@gmail.com>

Ben Finney wrote:
> Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:
> 
>> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>>> hehe. tab completion:
>> With bpython and ipython available, why would you even want to stick
>> to the 'plain old' interactive interpreter?
> 
> Because those optional extras are not always available, whereas the
> standard interactive interpreter is always available with CPython
> distributions.

I've never even contemplated the idea of installing 3rd party apps for
the 4 current development builds (2.x trunk, 2.x maintenance, 3.x trunk,
3.x maintenance) on my home development machine.

And of course work suffers from the problem of not being allowed to
install arbitrary apps I downloaded from the internet without getting
the licensing vetted for commercial acceptability.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 01:34:08 2010
From: nad at acm.org (Ned Deily)
Date: Sat, 16 Jan 2010 16:34:08 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
Message-ID: <nad-79FC16.16340816012010@news.gmane.org>

I've recently seen a couple of references to 3.1.2 go by in checkins 
which made me wonder whether dates have been proposed yet for updates to 
either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
references in the PEPs.  Some advance warning would be nice.  I assume 
that some critical problem could trigger planning for an update on short 
notice; is there a time-limit trigger as well?

-- 
 Ned Deily,
 nad at acm.org



From ncoghlan at gmail.com  Sun Jan 17 01:55:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:55:38 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52600A.7060201@gmail.com>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Usually there is one more regular maintenance release of the existing
maintenance branches within a few months of the creation of the next
version (releases before then are at the discretion of the release
manager, and security releases continue for much longer).

So take the planned 2.7 and 3.2 release dates and add a couple of months
to each one to get the likely release dates for the 2.6.x and 3.1.x
maintenance releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Sun Jan 17 10:21:49 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Jan 2010 10:21:49 +0100
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52D6AD.6090302@v.loewis.de>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Barry was once proposing time-based releases; this idea didn't really
catch on.

It's the release manager who decides when the next bug fix release will
be made, often in response to developers asking for one.

Regards,
Martin



From solipsis at pitrou.net  Sun Jan 17 14:00:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 13:00:04 +0000 (UTC)
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <loom.20100117T135910-313@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.

There are a couple of release blockers right now. Once they are fixed or
deferred, I think it would be nice to have a 3.1.2.
Why do you need "some advance warning" though?




From ncoghlan at gmail.com  Sun Jan 17 14:40:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 23:40:42 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <loom.20100117T135910-313@post.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
Message-ID: <4B53135A.7060104@gmail.com>

Antoine Pitrou wrote:
> Ned Deily <nad <at> acm.org> writes:
>> I've recently seen a couple of references to 3.1.2 go by in
>> checkins which made me wonder whether dates have been proposed yet
>> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
>> didn't see any references in the PEPs.  Some advance warning would
>> be nice.
> 
> There are a couple of release blockers right now. Once they are fixed
> or deferred, I think it would be nice to have a 3.1.2. Why do you
> need "some advance warning" though?

Advance warning does allow interested users that would consider
upgrading to schedule time for testing before the maintenance release
comes out. This is particularly useful in helping to make a 1-week RC
period effective in picking up issues that might otherwise lead to a
brown paper bag release to fix major issues that slipped through our own
automated test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 19:01:37 2010
From: nad at acm.org (Ned Deily)
Date: Sun, 17 Jan 2010 10:01:37 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
	<4B53135A.7060104@gmail.com>
Message-ID: <nad-25165C.10013717012010@ger.gmane.org>

In article <4B53135A.7060104 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> I've recently seen a couple of references to 3.1.2 go by in
> >> checkins which made me wonder whether dates have been proposed yet
> >> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
> >> didn't see any references in the PEPs.  Some advance warning would
> >> be nice.
> > There are a couple of release blockers right now. Once they are fixed
> > or deferred, I think it would be nice to have a 3.1.2. Why do you
> > need "some advance warning" though?
> 
> Advance warning does allow interested users that would consider
> upgrading to schedule time for testing before the maintenance release
> comes out. This is particularly useful in helping to make a 1-week RC
> period effective in picking up issues that might otherwise lead to a
> brown paper bag release to fix major issues that slipped through our own
> automated test coverage.

That. and resource contention: there are always potential fixes in the 
pipeline that could or should be bumped in priority if one knows there 
is a code cutoff approaching.

-- 
 Ned Deily,
 nad at acm.org



From ziade.tarek at gmail.com  Sun Jan 17 20:51:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 20:51:58 +0100
Subject: [Python-Dev] Enhancing the shutil module
Message-ID: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>

Hello,

For 2.7/3.2, I am in the process of removing modules in Distutils that
can be replaced by calls to existing functions in stdlib. For
instance, "dir_util" and "file_util" (old modules from the Python 1.x
era) are going away in favor of calls to shutil (and os), so the
Distutils package gets lighter.

Another module I would like to move away from Distutils is
"archive_util". It contains helpers to build archives, whether they
are zip or tar files. I propose to move those useful functions into
shutil, as this seems the most logical place.

I also propose to maintain this "shutil" module for now on (no one is
declared as a maintainer in maintainers.rst) since Distutils will
become a heavy user of its functions.

Any objections/opinions ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 20:53:03 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 19:53:03 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <4B536A9F.5050203@voidspace.org.uk>

On 17/01/2010 19:51, Tarek Ziad? wrote:
> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>
> Regards,
> Tarek
>
>    
I think it's a great idea. :-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From brett at python.org  Sun Jan 17 20:55:47 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jan 2010 11:55:47 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>

On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>

If it's archive-agnostic then shutil is probably the best place.


>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
>
Great!


> Any objections/opinions ?
>

No objections from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100117/f97ac6eb/attachment-0005.htm>

From solipsis at pitrou.net  Sun Jan 17 21:04:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 20:04:41 +0000 (UTC)
Subject: [Python-Dev] Enhancing the shutil module
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <loom.20100117T210307-719@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.

Are these functions portable? Do they rely on external programs?

> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.

It's ok for me.

Regards

Antoine.




From ziade.tarek at gmail.com  Sun Jan 17 21:09:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:09:18 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
Message-ID: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>

On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Hello,
>>
>> For 2.7/3.2, I am in the process of removing modules in Distutils that
>> can be replaced by calls to existing functions in stdlib. For
>> instance, "dir_util" and "file_util" (old modules from the Python 1.x
>> era) are going away in favor of calls to shutil (and os), so the
>> Distutils package gets lighter.
>>
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>
> If it's archive-agnostic then shutil is probably the best place.

In more details:
It allows the creation of gzip, bzip2, tar and zip files through a single API.
There's a registry of supported formats and the API is driven by a
format identifier.

To do the work it uses stdlib's compression modules. Although it tries
the "zip" system command as a fallback if the "zipfile" module is not
present.

(notice that I've removed the support of "compress" (.Z) some time ago)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 21:15:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 20:15:00 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <loom.20100117T210307-719@post.gmane.org>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
Message-ID: <4B536FC4.9090304@voidspace.org.uk>

On 17/01/2010 20:04, Antoine Pitrou wrote:
> Tarek Ziad?<ziade.tarek<at>  gmail.com>  writes:
>    
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>>      
> Are these functions portable? Do they rely on external programs?
>
>    

I believe that part of the work that Tarek has been doing has been to 
make these distutils commands use the Python standard library and not 
depend on external programs. In which case they seem like *excellent* 
additions to the shutil module.

Of course Tarek can speak for himself...

Michael

>> I also propose to maintain this "shutil" module for now on (no one is
>> declared as a maintainer in maintainers.rst) since Distutils will
>> become a heavy user of its functions.
>>      
> It's ok for me.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Sun Jan 17 21:43:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:43:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B536FC4.9090304@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
Message-ID: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:15 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>> Are these functions portable? Do they rely on external programs?
>>
>>
>
> I believe that part of the work that Tarek has been doing has been to make
> these distutils commands use the Python standard library and not depend on
> external programs. In which case they seem like *excellent* additions to the
> shutil module.

yes, in the past the "tar" files where created using the "tar" command
but this has been changed. For a while now, they are portable and they
use stdlib code only. A recent addition is to be able to define
user/group permissions in the tar files, thanks to Lars' work in the
tarfile module.

There's one remaining external call for "zip" done if the zip module
is not found, but I am happy to remove it and throw an exception if
it's not found, and keep the external "zip" call on Distutils side, so
shutil stays 100% stdlib-powered.

> Of course Tarek can speak for himself...

Thanks for explaining it ! :)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From sridharr at activestate.com  Sun Jan 17 22:50:52 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 17 Jan 2010 13:50:52 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
Message-ID: <4B53863C.5060304@activestate.com>

On 1/17/2010 12:09 PM, Tarek Ziad? wrote:
> On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon<brett at python.org>  wrote:
>> >  On Sun, Jan 17, 2010 at 11:51, Tarek Ziad?<ziade.tarek at gmail.com>  wrote:
>>> >>  Another module I would like to move away from Distutils is
>>> >>  "archive_util". It contains helpers to build archives, whether they
>>> >>  are zip or tar files. I propose to move those useful functions into
>>> >>  shutil, as this seems the most logical place.
>> >  If it's archive-agnostic then shutil is probably the best place.
> In more details:
> It allows the creation of gzip, bzip2, tar and zip files through a single API.
> There's a registry of supported formats and the API is driven by a
> format identifier.

Will it also allow decompression of the said archive types? Distribute 
has some utility code to handle zip/tar archives. So does PyPM. This is 
because the `tarfile` and `zipfile` modules do not "just work" due to 
several issues.

See http://gist.github.com/279606

Take note of the following in the above code:

  1) _ensure_read_write_access
  2) *File.is_valid
  3) ZippedFile.extract ... issue 6510
  4) ZippedFile.extract ... issue 6609
  5) TarredFile.extract ... issue 6584
  6) The way unpack() detects the unpacked directory.

-srid


From ziade.tarek at gmail.com  Sun Jan 17 23:09:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 23:09:29 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B53863C.5060304@activestate.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
	<4B53863C.5060304@activestate.com>
Message-ID: <94bdd2611001171409j4ec1e0bfj47e59894fdec0d8a@mail.gmail.com>

On Sun, Jan 17, 2010 at 10:50 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
[..]
> Will it also allow decompression of the said archive types?

No but it would make sense having this one as well.
Distribute/Setuptools' "unpack_archive" (used by easy_install) was
implemented using the same principle as Distutils' "make_archive".

I can add it as well while I am at it : it has been successfully used
for years by easy_install.

> Distribute has
> some utility code to handle zip/tar archives. So does PyPM. This is because
> the `tarfile` and `zipfile` modules do not "just work" due to several
> issues.
>
> See http://gist.github.com/279606
>
> Take note of the following in the above code:
>
> ?1) _ensure_read_write_access
> ?2) *File.is_valid
> ?3) ZippedFile.extract ... issue 6510
> ?4) ZippedFile.extract ... issue 6609
> ?5) TarredFile.extract ... issue 6584

Looks like some of these are already fixed (I just looked quickly in
the bug tracker though)
If its not already done and pending, it would be great if you could
refactor your fixes into patches for the remaining issues for each one
of those modules

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From barry at python.org  Sun Jan 17 23:56:37 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Jan 2010 17:56:37 -0500
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <4B52D6AD.6090302@v.loewis.de>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<4B52D6AD.6090302@v.loewis.de>
Message-ID: <BEC881FD-2BDE-4B5C-A570-B8C1E23D6DE1@python.org>

On Jan 17, 2010, at 4:21 AM, Martin v. L?wis wrote:

> Barry was once proposing time-based releases; this idea didn't really
> catch on.

I'm still a proponent of this, but it doesn't seem like there's enough support for it.

> It's the release manager who decides when the next bug fix release will
> be made, often in response to developers asking for one.

I'm happy to make a 2.6.5 release whenever it makes sense.
-Barry



From ncoghlan at gmail.com  Mon Jan 18 13:40:01 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Jan 2010 22:40:01 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
Message-ID: <4B5456A1.2080109@gmail.com>

Tarek Ziad? wrote:
> There's one remaining external call for "zip" done if the zip module
> is not found, but I am happy to remove it and throw an exception if
> it's not found, and keep the external "zip" call on Distutils side, so
> shutil stays 100% stdlib-powered.

+1 for that approach. These changes all sound like nice additions to
shutil, and hooray for every module that gets adopted by an active
maintainer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Mon Jan 18 14:34:14 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 14:34:14 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5456A1.2080109@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
Message-ID: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>

On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
> 
> Tarek Ziad? wrote:
>> There's one remaining external call for "zip" done if the zip module
>> is not found, but I am happy to remove it and throw an exception if
>> it's not found, and keep the external "zip" call on Distutils side, so
>> shutil stays 100% stdlib-powered.
> 
> +1 for that approach. These changes all sound like nice additions to
> shutil, and hooray for every module that gets adopted by an active
> maintainer :)

Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).

Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

From doug.hellmann at gmail.com  Mon Jan 18 14:46:05 2010
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 18 Jan 2010 08:46:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils  
>>> side, so
>>> shutil stays 100% stdlib-powered.
>>
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
>
> Isn't it a bit weird to include that to shutil though? shutil  
> advertises itself as "a number of high-level operations on files and  
> collections of files." and from what I understood it was a bunch of  
> shell-type utility functions to easily copy, move or remove files  
> and directories (that's pretty much all there is in it at this time).
>
> Wouldn't it make more sense to put those "archive utils" functions/ 
> objects in a new module separate from shutil, dealing specifically  
> with cross-archive APIs and linked from the current archive-specific  
> modules (essentially, just take the current archive_util, move it to  
> the toplevel of the stdlib and maybe rename it)? It would also make  
> the module much easier to find when searching through the module  
> listing, I think.

+1



From fuzzyman at voidspace.org.uk  Mon Jan 18 14:57:49 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 18 Jan 2010 13:57:49 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
Message-ID: <4B5468DD.5040200@voidspace.org.uk>

On 18/01/2010 13:46, Doug Hellmann wrote:
>
> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>
>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>
>>> Tarek Ziad? wrote:
>>>> There's one remaining external call for "zip" done if the zip module
>>>> is not found, but I am happy to remove it and throw an exception if
>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>> shutil stays 100% stdlib-powered.
>>>
>>> +1 for that approach. These changes all sound like nice additions to
>>> shutil, and hooray for every module that gets adopted by an active
>>> maintainer :)
>>
>> Isn't it a bit weird to include that to shutil though? shutil 
>> advertises itself as "a number of high-level operations on files and 
>> collections of files." 

Well - isn't what's being proposed "a number of high-level operations on 
files and collections of files." ?

>> and from what I understood it was a bunch of shell-type utility functions

like tar and zip? :-)

>> to easily copy, move or remove files and directories (that's pretty 
>> much all there is in it at this time).
>>

I don't think the additions are out of place prima-facie.

>> Wouldn't it make more sense to put those "archive utils" 
>> functions/objects in a new module separate from shutil, dealing 
>> specifically with cross-archive APIs and linked from the current 
>> archive-specific modules (essentially, just take the current 
>> archive_util, move it to the toplevel of the stdlib and maybe rename 
>> it)? It would also make the module much easier to find when searching 
>> through the module listing, I think.
>
> +1
>

Proliferation of modules is itself a bad thing though.

Michael


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Mon Jan 18 15:34:12 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 15:34:12 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <94bdd2611001180634sacd998fm6f67dc680e2e61c3@mail.gmail.com>

On Mon, Jan 18, 2010 at 2:57 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>>> Wouldn't it make more sense to put those "archive utils"
>>> functions/objects in a new module separate from shutil, dealing specifically
>>> with cross-archive APIs and linked from the current archive-specific modules
>>> (essentially, just take the current archive_util, move it to the toplevel of
>>> the stdlib and maybe rename it)? It would also make the module much easier
>>> to find when searching through the module listing, I think.
>>
>> +1
>>
>
> Proliferation of modules is itself a bad thing though.

I am with Michael here. I think having this function in shutil is the
right place.

For the find problem, I think docs.python.org is easy to search now,
as long as the shutil module documentation has an good set of examples
for this new API.

We could even add references in the tarfile, zipfile modules
documentation pointing to these examples.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org


From masklinn at masklinn.net  Mon Jan 18 15:56:05 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 15:56:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>

On 18 Jan 2010, at 14:57 , Michael Foord wrote:
> 
> On 18/01/2010 13:46, Doug Hellmann wrote:
>> 
>> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>> 
>>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>> 
>>>> Tarek Ziad? wrote:
>>>>> There's one remaining external call for "zip" done if the zip module
>>>>> is not found, but I am happy to remove it and throw an exception if
>>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>>> shutil stays 100% stdlib-powered.
>>>> 
>>>> +1 for that approach. These changes all sound like nice additions to
>>>> shutil, and hooray for every module that gets adopted by an active
>>>> maintainer :)
>>> 
>>> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." 
> 
> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
> 
Well no those are high-level operations on a very restricted set of file types (archives).

>>> and from what I understood it was a bunch of shell-type utility functions
> 
> like tar and zip? :-)
> 
Tar and zip have a module each at this time, so they're considered pretty big. I don't think anybody would consider "mv" big enough to give it a module on its own.

>>> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.
>> 
>> +1
>> 
> 
> Proliferation of modules is itself a bad thing though.
Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.

Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

From phd at phd.pp.ru  Mon Jan 18 16:03:38 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 18 Jan 2010 18:03:38 +0300
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <20100118150337.GA19391@phd.pp.ru>

On Mon, Jan 18, 2010 at 02:34:14PM +0100, Masklinn wrote:
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

   +1

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From ziade.tarek at gmail.com  Mon Jan 18 16:24:37 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 16:24:37 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
	<6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
Message-ID: <94bdd2611001180724u7f48e620ub32e13ef96c873b9@mail.gmail.com>

On Mon, Jan 18, 2010 at 3:56 PM, Masklinn <masklinn at masklinn.net> wrote:
[..]
>> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
>>
> Well no those are high-level operations on a very restricted set of file types (archives)

not really: make_archive/unpack_archive, are also dealing with files
and directories.

[..]
>> Proliferation of modules is itself a bad thing though.
> Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.
>
> Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

I am not sure why this would happen. For instance, shutil is already
on the top of the os module since a few major versions IIRC, because
it reads and writes files and directories. But it was not moved into
the os package (or vice-versa)

The shutil module uses APIs to read and write files. So if it works
with archives, it's just a specific read/write API that is used, but
that doesn't mean tarfile and zipfile might be reunited with shutil
imho.

If the shutil module is restricted to high-level files and directories
manipulation, working with archives is just a target like another I
think.

But at the end I am 0- to create a new module, because what really
matters to me is to take it out of Distutils :)

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From listsin at integrateddevcorp.com  Mon Jan 18 16:56:05 2010
From: listsin at integrateddevcorp.com (Steve Steiner (listsin))
Date: Mon, 18 Jan 2010 10:56:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>> 
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils side, so
>>> shutil stays 100% stdlib-powered.
>> 
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
> 
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

As much of a pain as it is to get new modules accepted, I agree that mixing archiving functions into shutil is not the right way to do it and that a separate archive_util module would make much more sense and would give a logical place to put any extensions to archive handling.

S





From rdmurray at bitdance.com  Mon Jan 18 20:28:47 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 18 Jan 2010 14:28:47 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
Message-ID: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>

On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
> As much of a pain as it is to get new modules accepted, I agree that
> mixing archiving functions into shutil is not the right way to do it
> and that a separate archive_util module would make much more sense and
> would give a logical place to put any extensions to archive handling.

Looking at the source code and API for both shutil and archive_util, I
think that the archive_util methods fit into shutil.  shutil currently
wraps some standard library facilities with convenience functions
for operations you might otherwise perform at the shell command line using
OS facilities.  As far as I can tell, archive_util does the same, and
seems quite within the shutil mission of "high level file operations".

So +1 from me for putting these in shutil.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From p.f.moore at gmail.com  Mon Jan 18 20:48:43 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 18 Jan 2010 19:48:43 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
Message-ID: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>

2010/1/18 R. David Murray <rdmurray at bitdance.com>:
> On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
>> As much of a pain as it is to get new modules accepted, I agree that
>> mixing archiving functions into shutil is not the right way to do it
>> and that a separate archive_util module would make much more sense and
>> would give a logical place to put any extensions to archive handling.
>
> Looking at the source code and API for both shutil and archive_util, I
> think that the archive_util methods fit into shutil. ?shutil currently
> wraps some standard library facilities with convenience functions
> for operations you might otherwise perform at the shell command line using
> OS facilities. ?As far as I can tell, archive_util does the same, and
> seems quite within the shutil mission of "high level file operations".
>
> So +1 from me for putting these in shutil.

Conceptually, I'm happy with these going into shutil (and +1 on the
rest of Tarek's proposal, too!)

To my mind, shutil is a module for higher-level operations on files -
the sort of things you'd do in shell commands, like move a batch of
files around (mv), create a directory tree (mkdir -p). Tarring or
zipping up a batch of files fits nicely into that space.

Paul.


From david.lyon at preisshare.net  Tue Jan 19 02:53:43 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 Jan 2010 20:53:43 -0500
Subject: [Python-Dev] PyCon Keynote
In-Reply-To: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
References: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
Message-ID: <0dfb370163cf3a284ca256f49662db74@preisshare.net>

On Wed, 13 Jan 2010 10:51:14 -0800, Guido van Rossum <guido at python.org>
wrote:
> Please mail me topics you'd like to hear me talk about in my keynote
> at PyCon this year.

As a typical (but perphaps more vocal) python developer I would
like to hear things that might aspire me and others to move to 
python 3.

The way I see it is that python 2.x represents everyones 
comfort zone. It's safe and nobody got fired at work for 
using python 2.x

I would like to hear how python 3 could be 'shaken' up
with a slightly less conservative policy that has gone
with the python 2.x series.

The logic perhaps being that since only a minority
use the 3.x series, it's functionality set is still
more up in the air. imo it needs bigger batteries
to give it more power than the 2.x series.

This meaning that the stdlib could take an extra
5-10 packages not in the 2.x series. Just as
one example, sphinx - a great documentation
tool. I can easily name five or six others.

I'd also like to hear more of your ideas on pypi
and getting it to have as much who-ha as CPAN.

You could do a lot to enlarge the developer
group. Help us all get our priorities to be
inline with your own wishes and expectations.

If you ask us all to put in a big year and
buy you a beer at the end.. I'm certain we
all will.

Every little bit of inspiration and direction
counts for a lot...

Best Regards

David











From ncoghlan at gmail.com  Tue Jan 19 12:20:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Jan 2010 21:20:05 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
	<79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
Message-ID: <4B559565.8090602@gmail.com>

Paul Moore wrote:
> 2010/1/18 R. David Murray <rdmurray at bitdance.com>:
>> So +1 from me for putting these in shutil.
> 
> Conceptually, I'm happy with these going into shutil (and +1 on the
> rest of Tarek's proposal, too!)
> 
> To my mind, shutil is a module for higher-level operations on files -
> the sort of things you'd do in shell commands, like move a batch of
> files around (mv), create a directory tree (mkdir -p). Tarring or
> zipping up a batch of files fits nicely into that space.

This is also reflected in the way at least Windows handles archives
these days - it took them a couple of iterations to get it right (and
resolve some of the performance impacts), but Explorer now does a decent
job of integrating archives into the directory tree as "folders that
happen to be compressed".

Are archives as fundamental as directories and files? No. But in the
context of shutil, the fact that their internal structure is largely
about directories and files makes them more than just another arbitrary
file type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 19 14:16:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 08:16:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
Message-ID: <20100119081639.5d431ed9@freewill>

I've just updated the Launchpad mirrors for the 4 active Python branches,
trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar branches
on code.python.org but it's probably been 7 months or so since those were
regularly updated.  Now the Launchpad branches sync against the read-only
Subversion branches at http://svn.python.org, so they should remain up-to-date
(within the re-sync timeframe of about 4 hours).

This means you can once again use Bazaar to get local branches of Python, and
you can of course push your own branches to Launchpad.  I believe you can even
use the bzr-svn plugin to commit changes back to the Subversion master, though
I have not yet tried this.

To get a local branch, just do any of the following:

    % bzr branch lp:python (for trunk)
    % bzr branch lp:python/2.6
    % bzr branch lp:python/py3k
    % bzr branch lp:python/3.1

(It's fairly easy to create new mirrors for other Subversion branches,
e.g. Python 2.5; just drop me an email if you want them.)

If you're going to create a lot of branches you probably want to put them in a
shared repository.  E.g.

    % bzr init-repo pythonbzr
    % cd pythonbzr
    % bzr branch lp:python/py3k

Bazaar 2.0 or better is recommended.  For me, it took about 5m to check the
first branch out from Launchpad, and then about 30s or so for each subsequent
branch.

Enjoy,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/89a42d75/attachment-0003.pgp>

From abpillai at gmail.com  Tue Jan 19 14:37:18 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Tue, 19 Jan 2010 19:07:18 +0530
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <8548c5f31001190537l2bcab5cfkb6f461910e4abd8b@mail.gmail.com>

On Mon, Jan 18, 2010 at 1:21 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>

+1 for this. Just make sure that you change the docstring of shutil
 which now reads as,

" shutil - Utility functions for copying files and directory trees."

 According to this "definition", archives don't fit in there. But the
 functionality does fit right in, so just need to make sure that it
 is reflected in the __doc__ .


> Regards,
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/55892759/attachment-0003.htm>

From vinay_sajip at yahoo.co.uk  Tue Jan 19 16:50:42 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 19 Jan 2010 15:50:42 +0000 (UTC)
Subject: [Python-Dev] Mailing List archive corruption?
Message-ID: <loom.20100119T164757-651@post.gmane.org>

Hi,

When I look at the mailing list archive for python-dev, I see some odd stuff at
the bottom of the page:

http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232

Anyone know what's happened?

Regards,

Vinay Sajip



From barry at python.org  Tue Jan 19 17:07:48 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:07:48 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <loom.20100119T164757-651@post.gmane.org>
References: <loom.20100119T164757-651@post.gmane.org>
Message-ID: <20100119110748.69bc564a@freewill>

On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:

>When I look at the mailing list archive for python-dev, I see some odd stuff at
>the bottom of the page:
>
>http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>
>Anyone know what's happened?

WTF?  I think the archives were recently regenerated, so there's probably a
fubar there.  CC'ing the postmasters.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/8947a2e9/attachment-0003.pgp>

From foom at fuhm.net  Tue Jan 19 17:24:57 2010
From: foom at fuhm.net (James Y Knight)
Date: Tue, 19 Jan 2010 11:24:57 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119110748.69bc564a@freewill>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
Message-ID: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>


On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:

> On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
>
>> When I look at the mailing list archive for python-dev, I see some  
>> odd stuff at
>> the bottom of the page:
>>
>> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>>
>> Anyone know what's happened?
>
> WTF?  I think the archives were recently regenerated, so there's  
> probably a
> fubar there.  CC'ing the postmasters.

That happens if messages had unescaped "From" lines in the middle of  
them.

No doubt, you've now broken every link anyone had ever made into the  
python-dev archives, because now all the article numbers are  
different. BTDT...unfortunately... Pipermail really is quite crappy,  
sigh.

Anyhow, when I did that, I went back to a backup to get the original  
article numbers, and edited the mbox file escaping From lines or  
adding additional empty messages until the newly regenerated article  
numbers matched the originals. I'd highly recommend going through that  
painful process, since I suspect a *lot* of people have links to the  
python-dev archive. Hope you have a backup (or can find caches on  
google or archive.org or something).

James


From barry at python.org  Tue Jan 19 17:44:21 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:44:21 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119114421.3b96fbd4@freewill>

On Jan 19, 2010, at 11:24 AM, James Y Knight wrote:

>No doubt, you've now broken every link anyone had ever made into the  
>python-dev archives, because now all the article numbers are  
>different. BTDT...unfortunately... Pipermail really is quite crappy,  
>sigh.

I've been trying for 10+ years to get folks interested in helping me fix this
(and a few other warts) in Pipermail, to not much success. ;/

>Anyhow, when I did that, I went back to a backup to get the original  
>article numbers, and edited the mbox file escaping From lines or  
>adding additional empty messages until the newly regenerated article  
>numbers matched the originals. I'd highly recommend going through that  
>painful process, since I suspect a *lot* of people have links to the  
>python-dev archive. Hope you have a backup (or can find caches on  
>google or archive.org or something).

bin/cleanarch uses a set of heuristics to find unescaped From lines and fix
them.  It's generally pretty good, but it certain can change message numbers
(and sadly, their urls).

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/e32aa882/attachment-0003.pgp>

From rdmurray at bitdance.com  Tue Jan 19 18:48:41 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Jan 2010 12:48:41 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119174841.9BC3B16A53@kimball.webabinitio.net>

On Tue, 19 Jan 2010 11:24:57 -0500, James Y Knight <foom at fuhm.net> wrote:
> 
> On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:
> 
> > On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
> >
> >> When I look at the mailing list archive for python-dev, I see some  
> >> odd stuff at the bottom of the page:
> >>
> >> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
> >>
> >> Anyone know what's happened?
> >
> > WTF?  I think the archives were recently regenerated, so there's  
> > probably a fubar there.  CC'ing the postmasters.
> 
> That happens if messages had unescaped "From" lines in the middle of  
> them.
> 
> No doubt, you've now broken every link anyone had ever made into the  
> python-dev archives, because now all the article numbers are  
> different. BTDT...unfortunately... Pipermail really is quite crappy,  
> sigh.
> 
> Anyhow, when I did that, I went back to a backup to get the original  
> article numbers, and edited the mbox file escaping From lines or  
> adding additional empty messages until the newly regenerated article  
> numbers matched the originals. I'd highly recommend going through that  
> painful process, since I suspect a *lot* of people have links to the  
> python-dev archive. Hope you have a backup (or can find caches on  
> google or archive.org or something).

The Python issue tracker does, for one.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From ncoghlan at gmail.com  Tue Jan 19 22:18:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 07:18:52 +1000
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119174841.9BC3B16A53@kimball.webabinitio.net>
References: <loom.20100119T164757-651@post.gmane.org>	<20100119110748.69bc564a@freewill>	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
	<20100119174841.9BC3B16A53@kimball.webabinitio.net>
Message-ID: <4B5621BC.4070608@gmail.com>

R. David Murray wrote:
> The Python issue tracker does, for one.

And all the PEPs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From david.lyon at pythontest.org  Wed Jan 20 00:16:44 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 10:16:44 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119081639.5d431ed9@freewill>
References: <20100119081639.5d431ed9@freewill>
Message-ID: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>


Hi Barry,

That looks very interesting...

So does that mean we could update the stdlib for a given
python version using this ?

David

> I've just updated the Launchpad mirrors for the 4 active Python branches,
> trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar
> branches
> on code.python.org but it's probably been 7 months or so since those were
> regularly updated.  Now the Launchpad branches sync against the read-only
> Subversion branches at http://svn.python.org, so they should remain
> up-to-date
> (within the re-sync timeframe of about 4 hours).
>
> This means you can once again use Bazaar to get local branches of Python,
> and
> you can of course push your own branches to Launchpad.  I believe you can
> even
> use the bzr-svn plugin to commit changes back to the Subversion master,
> though
> I have not yet tried this.
>
> To get a local branch, just do any of the following:
>
>     % bzr branch lp:python (for trunk)
>     % bzr branch lp:python/2.6
>     % bzr branch lp:python/py3k
>     % bzr branch lp:python/3.1
>
> (It's fairly easy to create new mirrors for other Subversion branches,
> e.g. Python 2.5; just drop me an email if you want them.)
>
> If you're going to create a lot of branches you probably want to put them
> in a
> shared repository.  E.g.
>
>     % bzr init-repo pythonbzr
>     % cd pythonbzr
>     % bzr branch lp:python/py3k
>
> Bazaar 2.0 or better is recommended.  For me, it took about 5m to check
> the
> first branch out from Launchpad, and then about 30s or so for each
> subsequent
> branch.
>
> Enjoy,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/david.lyon%40pythontest.org
>




From barry at python.org  Wed Jan 20 00:54:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 18:54:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119185439.299660c7@freewill>

On Jan 20, 2010, at 10:16 AM, David Lyon wrote:

>Hi Barry,
>
>That looks very interesting...

Hi David,

>So does that mean we could update the stdlib for a given
>python version using this ?

In a sense, yes (if I understand your question correctly).

You can use Bazaar to branch any of the 4 Python series and use all the modern
DVCS goodness to develop your updates.  You can share your branches with
others via e.g. Launchpad and even request reviews (called "merge proposals")
to get feedback from others.

The one thing I am unsure about, mostly because I have not tried it, is
whether your Bazaar branch can be used to commit directly back to the Python
Subversion master branches.  I /think/ the answer is yes, assuming of course
that you have permission to do so, and that you have a modern version of
Bazaar and the bzr-svn plugin.

It might even be possible to commit your Bazaar branch to a local Subversion
branch, and then commit the latter to get it pushed up to svn.python.org.

At worst, you would use Bazaar's features to get your patch into a state you
and your reviewers are happy with, then you would generate a diff for
application to your copy of the svn.python.org branch.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/37753b1a/attachment-0003.pgp>

From david.lyon at pythontest.org  Wed Jan 20 01:51:23 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 11:51:23 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119185439.299660c7@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
Message-ID: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>

> On Jan 20, 2010, at 10:16 AM, Barry wrote:

>> So does that mean we could update the stdlib for a given
>> python version using this ?
>
> In a sense, yes (if I understand your question correctly).

Yeah, it just needs an implementation.

> The one thing I am unsure about, mostly because I have not tried it, is
> whether your Bazaar branch can be used to commit directly back to the
> Python Subversion master branches.  I /think/ the answer is yes,
> assuming of course that you have permission to do so...

Well I'm too Senior and my stuff is too forward looking to qualify
for that just yet.

I'd be happy to see bzr and mercurial and git all made it together
into the stdlib for python 3. That would give a superb updating
mechanism for python that would propel python well beyond
the dinosaur badlands of CPAN and other languages.

I was actually reading from
(http://en.wikipedia.org/wiki/Python_%28programming_language%29):

"Rather than requiring all desired functionality to be built into the
language's core, Python was designed to be highly extensible. .. .. This
design of a small core language with a large standard library and an
easily extensible interpreter was intended by Van Rossum from the very
start because of his frustrations with ABC (which espoused the opposite
mindset).[5]"

To me, the source code control systems seem to be fully in tune
with the original design of python. That is, to be able to
easily pull external libraries in.

I think what has changed is that the mechanisms now (the SCMs)
are way more highly developed than before. Apart from that
though, after reading the full wikipedia article I'm left
with the distinct impression that things are still pretty
much the same (in that python design philosophy is advanced),
just that the landscape (of external C libraries) has changed.

Now all the libraries are external (on the internet) and
all externally managed.

So with just a tiny amount of work, imho we could pull
it all together to bring python 3 *back* to being that
cool tool that it once was (not saying it isn't now).

Were you offering me an experimental branch somewhere
for python 3 SCM integration ?

David







From jnoller at gmail.com  Wed Jan 20 02:09:15 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 20:09:15 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>

On Tue, Jan 19, 2010 at 7:51 PM, David Lyon <david.lyon at pythontest.org> wrote:
>> On Jan 20, 2010, at 10:16 AM, Barry wrote:
>
>>> So does that mean we could update the stdlib for a given
>>> python version using this ?
>>
>> In a sense, yes (if I understand your question correctly).
>
> Yeah, it just needs an implementation.
>
>> The one thing I am unsure about, mostly because I have not tried it, is
>> whether your Bazaar branch can be used to commit directly back to the
>> Python Subversion master branches. ?I /think/ the answer is yes,
>> assuming of course that you have permission to do so...
>
> Well I'm too Senior and my stuff is too forward looking to qualify
> for that just yet.
>
> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

i sincerely doubt that a source control system will be included in the
standard library in the future. Especially 3. A SCM is not a "package
management system".

Barry was talking about mirrors of the python code. It is true a
"package manager" could be developed based on a SCM, however you need
to implement this far away from the stdlib and get traction with it
within the community long before inclusion would be considered.

The decision to move python's source control from SVN to mercurial was
controversial enough; including 3 or more scm libraries into core
would be an intractable uphill mountain of bike sheds.

> So with just a tiny amount of work, imho we could pull
> it all together to bring python 3 *back* to being that
> cool tool that it once was (not saying it isn't now).

Python 3 is still modularized, still has a standard library, etc. If
you're really interested in helping with the standard library, get on
stdlib-sig, and get ready to write code and PEPs.

> Were you offering me an experimental branch somewhere
> for python 3 SCM integration ?

Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

jesse


From barry at python.org  Wed Jan 20 04:32:30 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 22:32:30 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <20100119223230.4c4a7ed5@freewill>

On Jan 19, 2010, at 08:09 PM, Jesse Noller wrote:

>The decision to move python's source control from SVN to mercurial was
>controversial enough; including 3 or more scm libraries into core
>would be an intractable uphill mountain of bike sheds.

I'd be surprised if any of the big 3 DVCS developers would actually /want/
their stuff in the stdlib.  Being in the stdlib has its advantages and
disadvantages.  I think for rapidly developing technology, the latter can
actually outweigh the former.

(Besides, git in the stdlib doesn't make much sense :).

>Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

Bingo.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/9ef0ed2b/attachment-0003.pgp>

From david.lyon at pythontest.org  Wed Jan 20 04:43:12 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 14:43:12 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>


> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

> Python 3 is still modularized, still has a standard library, etc. If
> you're really interested in helping with the standard library, get on
> stdlib-sig, and get ready to write code and PEPs.

Thank you for your direction to move these items forward to PEPs
and Code.

> i sincerely doubt that a source control system will be included in the
> standard library in the future. Especially 3.

Yeah and who twenty years ago thought you would get a 1GB memory
card for $3 when all we had was 10Meg hard disks and they were
the full 8" platter.

> A SCM is not a "package management system".

Exactly. It almost makes the need for a "package management system"
pretty much obsolete if you can update your code directly from
the developers sources.

That's what all these SCMs provide. Plus it's addictive. It's
hard to go back to 'package' style technology once you have
all your code on an SCM based feed.

> Barry was talking about mirrors of the python code. It is true a
> "package manager" could be developed based on a SCM, however you need
> to implement this far away from the stdlib and get traction with it
> within the community long before inclusion would be considered.

I think I'll have better chances with PEPs.

Being honest, if wonderful libraries like Sphinx and Mercurial
and Git and BZR can't make it into the stdlib, then there is
no hope for even newer code to get in there.

Plus, promoting all sorts of new and fangled tools however
good they may or may not be just confuses users and ends
up being a waste of time imho. It isn't good management
of volounteers time and effort.

If you could imagine disaster relief coordinated this
way, it would just be a disaster in itself.

That's why it has taken some 5 years to get PEP-345 done.

> The decision to move python's source control from SVN to mercurial was
> controversial enough; including 3 or more scm libraries into core
> would be an intractable uphill mountain of bike sheds.

Not at all.

It would be a very fair thing to do. Not to mention being
great for users.

> Barry made bzr mirrors of the python svn tree. Not a python with bzr
> included.

I can't resist asking for that again.. I heard it only in Monty
speek. Did you just say ?:

 "Barry made a bizarre mirror of the python suvern tree. Not a
  python with a buzzer included."

Anyway.. Maybe I do get what your talking about. Even if you do
talk with a strange accent. :-)

David






From jnoller at gmail.com  Wed Jan 20 05:10:07 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 23:10:07 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001192010v66b728dbxa7ad1ed1fc1df15f@mail.gmail.com>

On Tue, Jan 19, 2010 at 10:43 PM, David Lyon <david.lyon at pythontest.org> wrote:
[snip]
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.

Did you ever stop to think that some package authors do not want their
code in the standard library? That throwing random shiny things in
there just makes it a junk drawer?

Besides: Show us the PEP to include Sphinx, and it's dependencies in
the standard lib, with Georg's signature at the bottom.

The authors of modules have to want things to be in there, they have
to be best of breed, tested or common-enough patterns to warrant a
slot. We have things in the standard lib which still need more TLC and
maintenance, or they need to be shunted into space.

Adding random things, which may or may not help packaging, and
installing things from random places in someone source repository
won't make things better.

> Plus, promoting all sorts of new and fangled tools however
> good they may or may not be just confuses users and ends
> up being a waste of time imho. It isn't good management
> of volounteers time and effort.
>
> If you could imagine disaster relief coordinated this
> way, it would just be a disaster in itself.
>
> That's why it has taken some 5 years to get PEP-345 done.

I'm going to assume that you're trolling now, or intentionally
misrepresenting facts. Maybe a little of both. A PEP, and an
implementation and the ability to rationally debate, discuss and
defend your proposal is what is needed to enact changes on policies,
python-core or the standard library.

>> The decision to move python's source control from SVN to mercurial was
>> controversial enough; including 3 or more scm libraries into core
>> would be an intractable uphill mountain of bike sheds.
>
> Not at all.
>
> It would be a very fair thing to do. Not to mention being
> great for users.

There should be one-- and preferably only one --obvious way to do it.

> Anyway.. Maybe I do get what your talking about. Even if you do
> talk with a strange accent. :-)

My sense of humor has been disabled by repeated stunning at your
hands. I admire your enthusiasm, even if I do think some of it is
misplaced, or at guided into the proper channels at very least.

Please, you seem to have the time and willingness to help, please go
about this the right way. Discuss things on the proper lists, make
concrete proposals. If you have have standard lib changes, discuss
them on stdlib-sig, if you have ideas about python-the-language, or
the interpreter, etc - please discuss it on python-ideas.

jesse


From ben+python at benfinney.id.au  Wed Jan 20 05:16:25 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 15:16:25 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87wrzdfm1y.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
> and BZR can't make it into the stdlib, then there is no hope for even
> newer code to get in there.

Those are applications, not libraries. Applications don't belong in the
standard library.

-- 
 \     ?If you pick up a starving dog and make him prosperous, he will |
  `\      not bite you. This is the principal difference between a dog |
_o__)                    and a man.? ?Mark Twain, _Pudd'n'head Wilson_ |
Ben Finney



From brian.curtin at gmail.com  Wed Jan 20 05:19:47 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 19 Jan 2010 22:19:47 -0600
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <cf9f31f21001192019p25665318t3d99caf3db27198e@mail.gmail.com>

On Tue, Jan 19, 2010 at 21:43, David Lyon <david.lyon at pythontest.org> wrote:

>
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
>

I'm not entirely sure I see why the inclusion of a SCM into the stdlib is
necessary.

Just because pieces of software are mature and proven in their fields
doesn't mean we should add them, or that them *not* being in the stdlib
should be a basis for other projects making it in.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/3a2d80f8/attachment-0003.htm>

From barry at python.org  Wed Jan 20 05:26:44 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:26:44 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119232644.7fa25691@freewill>

On Jan 20, 2010, at 02:43 PM, David Lyon wrote:

>> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

>> A SCM is not a "package management system".
>
>Exactly. It almost makes the need for a "package management system"
>pretty much obsolete if you can update your code directly from
>the developers sources.
>
>That's what all these SCMs provide. Plus it's addictive. It's
>hard to go back to 'package' style technology once you have
>all your code on an SCM based feed.

Well...  I'm not so sure.  A package management system like apt does a /ton/
of additional bookkeeping and work to ensure a robust, highly consistent,
functioning system.  And while both Python and most Linux distributions have
their own notion of "package management", they don't always play nicely
together.  Tarek and the distutils-sig's work is trying to make the world a
better place by bridging this gap better, and there is code out there that
makes it easier to say import a Python package from the Cheeseshop and
.deb-ify it for use on Debian and Ubuntu.

There's also work being done in Launchpad that will allow you to
"build-from-branch" so that in a sense you could let a build farm take your
Bazaar branches and automatically build the packages from them.

I've strayed off-topic I suppose, but I see SCMs and package managers as
complementary technologies that help with important parts of the process of
delivering software to end-users, but I don't quite see how one can make the
other obsolete.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/4be8f756/attachment-0003.pgp>

From david.lyon at pythontest.org  Wed Jan 20 05:29:34 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 15:29:34 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>

> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>
> I'd be surprised if any of the big 3 DVCS developers would actually /want/
> their stuff in the stdlib.

If they ask, they'll get told they're motorbike-shedding. "It's better
if their users ask". So here I am as a user doing things the 'right'
way.

> Being in the stdlib has its advantages and
> disadvantages.  I think for rapidly developing technology, the
> latter can actually outweigh the former.

If it's about being able to do updates, then I think this
resolves an old and circular argument. As the SCM implementation
would, one would expect, to be able to update itself.

Side benefits are that it can update everything else along
with it at the same time. User Apps, Packages, whatever.

It's even better having SCM in an Industrial/Scientific
environment. Here's an example:

 - a machine breaks..  (I mean the software for/in it)

 - you fix the code, maybe on the spot

 - you commit and push back to the repository

 - your code gets checked in and run through the testbot
   and then you get blamed and have to do the whole thing
   again properly with a test case. Oh well..

Well anyway, whatever you guys might say, that's a whole
lot more efficient than running back to the development
machine and going through some obscure build and test
and publish process to do a fix on a production machine.

Point : The fact that SCMs are two way is great in
        a production environment. No packaging solution
        can come close.

So why not have python SCMs included as batteries in python..

All these arguments I can take off to the stdlib list when I get
the chance..

David




From barry at python.org  Wed Jan 20 05:50:36 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:50:36 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119235036.57f6e867@freewill>

Okay, last follow up on this and then I'm going to bed. :)

On Jan 20, 2010, at 03:29 PM, David Lyon wrote:

>> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>>
>> I'd be surprised if any of the big 3 DVCS developers would actually /want/
>> their stuff in the stdlib.
>
>If they ask, they'll get told they're motorbike-shedding. "It's better
>if their users ask". So here I am as a user doing things the 'right'
>way.

Actually, you're not.  It's not up to the Python community to initiate this.
If you really want this, you should engage with the relevant DVCS communities
and push them to request it.

>Side benefits are that it can update everything else along
>with it at the same time. User Apps, Packages, whatever.

I get that.  Heck, I still run one Gentoo server which I think is as close to
the edge you're describing as I'm comfortable with.  It's all great until the
wheels come off and then it can take *days* to get a functioning system
again.

The big difference is that I rely on my DVCS to keep one small thing, or a few
variants of the same thing, all sane.  But I rely on my distribution vendor to
keep a thousand complex, interdependent, interacting, sometimes conflicting
things sane and working.

>Point : The fact that SCMs are two way is great in
>        a production environment. No packaging solution
>        can come close.

Try talking with some hard-core operations guys, the folks with the keys to
the data centers, who work tireless, insanely hours keeping incredibly complex
systems running with very little downtime.  I think you'd get a different
perspective to put it mildly. :)

to-sleep-perchance-to-dream-ly y'rs,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/0def1cc9/attachment-0003.pgp>

From david.lyon at pythontest.org  Wed Jan 20 06:10:15 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 16:10:15 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <87wrzdfm1y.fsf@benfinney.id.au>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
Message-ID: <1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>

> "David Lyon" <david.lyon at pythontest.org> writes:
>
>> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
>> and BZR can't make it into the stdlib, then there is no hope for even
>> newer code to get in there.
>
> Those are applications, not libraries. Applications don't belong in the
> standard library.

Haha funny..

Well using that logic, distutils is an application..

Are you saying that distutils should be removed? That
is most certainly an application.

Lets not get too pedantic here. Mercurial and bzr have a built
in API that can be called in a library like way. It's true they
also have a command line interface in the same way that distutils
does.

I'm not saying anything negative about distutils. Given that
Tarek has an upcoming Pycon presentation where the program
talks about a distutils revamp.

I'm hoping that he can find some young 20 yr olds and
put a cool web interface on that thing. Given that there
are empty sprints at pycon. It couldn't hurt to throw
that challenge out. Anyway, we'll see..


David




From ben+python at benfinney.id.au  Wed Jan 20 06:32:10 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:32:10 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
	<20100119235036.57f6e867@freewill>
Message-ID: <87iqaxfijp.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> On Jan 20, 2010, at 03:29 PM, David Lyon wrote:
> >So here I am as a user doing things the 'right' way.
>
> Actually, you're not. It's not up to the Python community to initiate
> this. If you really want this, you should engage with the relevant
> DVCS communities and push them to request it.

Where ?push? must be strictly limited by a continual awareness that the
whole idea could just be bad.

If you find yourself in a tiny minority pushing for a change, it *could*
be that you have a great idea and the vast majority don't realise it
yet. But you must be realistic about the likelihood that the change is a
very *bad* idea, and frequently evaluate it for signs of that.

-- 
 \     ?I used to think that the brain was the most wonderful organ in |
  `\   my body. Then I realized who was telling me this.? ?Emo Philips |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Wed Jan 20 06:34:07 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:34:07 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
	<1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87eillfigg.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Well using that logic, distutils is an application..

Distutils is an application, the function of which is essential to
allowing sane development of Python packages. It's a special case. We
need to strictly limit the number of special cases, not gleefully add to
them.

-- 
 \        ?I'd take the awe of understanding over the awe of ignorance |
  `\                                          any day.? ?Douglas Adams |
_o__)                                                                  |
Ben Finney



From matthieu.brucher at gmail.com  Wed Jan 20 07:49:36 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 07:49:36 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>

> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

I think there are several points that make them not includable in Python:
- git is not written in Python
- bzr and mercurial have a life cycle much shorter than Python's, it's
the same issue than with other libraries where another community
develops them.

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From david.lyon at pythontest.org  Wed Jan 20 08:20:02 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 18:20:02 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
Message-ID: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>


Matthieu,

>> I'd be happy to see bzr and mercurial and git all made it together
>> into the stdlib for python 3. That would give a superb updating
>> mechanism for python that would propel python well beyond
>> the dinosaur badlands of CPAN and other languages.
>
> I think there are several points that make them not includable in Python:
> - git is not written in Python
> - bzr and mercurial have a life cycle much shorter than Python's, it's
> the same issue than with other libraries where another community
> develops them.

That's only two points. :-)

On 1; If that's true, I won't mention git again.

On 2; Who knows what their life cycle is. CVS is pretty much
      dead, and svn looks like it is on the way out.
      I can't think of how anything could be better than
      mercurial or bzr but I know I will be proved wrong.

At the end of the day, we are making a decision about whether
the language is 'set-in-stone' or whether it is still
evolving.

To me, Python 1.x had it's own distinct "era", as has
Python 2.x

Hoping that the Python 3 era can be a little more flexible
and perphaps "cleaner" than the 2.x era is all that I am
thinking here.

Have a nice day

David





From matthieu.brucher at gmail.com  Wed Jan 20 09:05:19 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 09:05:19 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001200005j6344672fo99826cc2e8648141@mail.gmail.com>

> That's only two points. :-)

In French, we say that several starts with 2 ;)

> On 1; If that's true, I won't mention git again.

I tis, you can check on the git repository (it's a mix of C, perl,
shell scripts, Python, ...)

> On 2; Who knows what their life cycle is.

You can check on their websites, their cycles are far shorter than
Python minor releases (several months vs several years).

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From abpillai at gmail.com  Wed Jan 20 10:36:53 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Wed, 20 Jan 2010 15:06:53 +0530
Subject: [Python-Dev] E3 BEFEHLE
In-Reply-To: <e9764b731001162012y519de6dbk74aab0930e3cf18c@mail.gmail.com>
References: <hit9gr$hal$1@ger.gmane.org>
	<b8e622741001161932n15a0e0d6i69ec06c1f820e6b@mail.gmail.com>
	<e9764b731001162012y519de6dbk74aab0930e3cf18c@mail.gmail.com>
Message-ID: <8548c5f31001200136r54bc9e2aw49485280864ecb2d@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:42 AM, Dj Gilcrease <digitalxero at gmail.com> wrote:

> 2010/1/16 Jack Diederich <jackdied at gmail.com>:
> > Good lord, did this make it past other people's spam filters too?  I
> > especially liked the reference to "REGION -2,0 ; Rlyeh".  Ph'nglui
> > mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn to you too sir.
>
> Ya made it past mine too, it looks like a debug dump of a macro for a
> some German game based either LOTR or Cthulhu
>

I initially thought it was  a Python disassembler trace of some step
of operations which failed, converted to German.

In fact, I was looking for a question at the end regarding REPL.
How very optimistic...


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100120/5dd68498/attachment-0003.htm>

From stephen at xemacs.org  Wed Jan 20 11:22:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 20 Jan 2010 19:22:55 +0900
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <87aaw9gjnk.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > (Besides, git in the stdlib doesn't make much sense :).

"Dulwich."


From solipsis at pitrou.net  Wed Jan 20 12:28:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Jan 2010 11:28:16 +0000 (UTC)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100120T122742-162@post.gmane.org>

David Lyon <david.lyon <at> pythontest.org> writes:
> 
> I think I'll have better chances with PEPs.
> 
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
> [snip]

This is python-ideas material, can you take it there? Thank you.

Antoine.




From ncoghlan at gmail.com  Wed Jan 20 13:16:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 22:16:34 +1000
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>	<20100119185439.299660c7@freewill>	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B56F422.5010607@gmail.com>

David Lyon wrote:
> On 2; Who knows what their life cycle is. CVS is pretty much
>       dead, and svn looks like it is on the way out.
>       I can't think of how anything could be better than
>       mercurial or bzr but I know I will be proved wrong.

I believe you misunderstood what Matthieu meant by life cycle there:
think "release cycle". If a project pushes out new releases
significantly more often than every 18-24 months (as is currently true
for all of the major SCM tools), then that fact alone makes it a very
bad fit for the Python standard library.

And centralised source control will be going strong for years. The DVCS
approach may be great for the open source world, but the gains are far
more limited in a closed source shop (especially a group writing
internal corporate applications which doesn't need to keep many, if any,
maintenance branches going).

If we weren't dealing with 4 active branches, the DVCS discussion would
have got a lot less traction with the core developers - aside from
better handling of multiple lines of development, most of the benefits
of the switch to a DVCS accrue to people without commit access to the
SVN repository.

Anyway, we've wandered far afield from legit python-dev topics now. Any
further ideas about super_mega_easy_install functionality that can pull
code from source control systems and build it rather than requiring
prebuild source tarballs should be directed to python-ideas (they
probably need to bake more even before they make an appearance on
distutils-sig).

Cheers,
Nick.

P.S. As Jesse said... your enthusiasm is great, but please don't assume
that some inherent conservatism on the part of other developers is
automatically evil or the result of a failure to see your point. A lot
of people around the world rely on our stuff every day. We owe it to
them to be measured in our actions and to put serious thought into any
major changes or additions we make to the language and the standard
library. For the current stage of its development, Python 3 is in a good
place from our point of view - its major carrot has really always been
the better Unicode support it offers, and the ever-increasing
globalisation of the web will create more and more pressure pushing
developers in that direction as the years go by. Sure, Python 3 cleans
up assorted other things as well, but the change to the text processing
model is the big one that is fundamentally incompatible with the
architecture of the 2.x series. Compared to that change, everything else
is just tinkering.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ziade.tarek at gmail.com  Fri Jan  1 16:07:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:07:20 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04D81D51@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<94bdd2610912141458m52da21ear4970506a37214e6@mail.gmail.com>
	<4dab5f760912230700l38a597f7l855bb2304025d6d5@mail.gmail.com>
	<F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
Message-ID: <94bdd2611001010707h4043ff64x7476049e435852b@mail.gmail.com>

2009/12/23 Glyph Lefkowitz <glyph at twistedmatrix.com>:
[..]
> 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages".
>
> The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. ?In other words it's got "bin", "lib", "share", "etc", etc.. ?~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation.

Right, good idea

Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Fri Jan  1 16:32:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:32:27 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
Message-ID: <94bdd2611001010732o38a563bcu6d0cc9122ed5c88f@mail.gmail.com>

On Sat, Dec 12, 2009 at 9:02 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> A while ago I've proposed to refactor the APIs that provides access to
> the installation paths and configuration variables at runtime into a
> single module called "sysconfig", and make it easier for all
> implementations to work with them.

I've applied the suggestions made in this thread.

If no one objects, here's what I am going to do now:

I'll merge this change in the trunk, (I'll drop the branch changesets
in favor of one single, clean changeset) and start to work on the
corresponding doc, so more people will be able to give some feedback
on the APIs once the second 2.7 alpha is out.

Then, in a second step, I'll work on the removal of the Makefile and
pyconfig.h dependencies by adding a _synconfig.py file as suggested
earlier, that will be created during the build process.

Regards,
Tarek


From status at bugs.python.org  Fri Jan  1 18:07:11 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jan 2010 18:07:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100101170711.A5AAF78654@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/25/09 - 01/01/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2537 open (+22) / 16902 closed (+19) / 19439 total (+41)

Open issues with patches:  1016

Average duration of open issues: 705 days.
Median duration of open issues: 462 days.

Open Issues Breakdown
   open  2504 (+22)
pending    32 ( +0)

Issues Created Or Reopened (42)
_______________________________

doc: Code is not always colored                                  12/30/09
CLOSED http://bugs.python.org/issue7487    reopened flox                          
                                                                               

documention buglet for PyBuffer                                  12/26/09
CLOSED http://bugs.python.org/issue7577    created  ronaldoussoren                
       easy                                                                    

Behavior of operations on a closed file object is not documented 12/26/09
       http://bugs.python.org/issue7578    created  blep                          
                                                                               

Patch to add docstrings to msvcrt                                12/26/09
       http://bugs.python.org/issue7579    created  brian.curtin                  
       patch                                                                   

distutils makefile from python.org installer doesn't work on OS  12/27/09
       http://bugs.python.org/issue7580    created  bskaplan                      
                                                                               

incomplete doc of zlib                                           12/27/09
CLOSED http://bugs.python.org/issue7581    created  coolwanglu                    
                                                                               

[patch] diff.py to use iso timestamp                             12/27/09
       http://bugs.python.org/issue7582    created  techtonik                     
       patch                                                                   

doctest should normalize tabs when comparing output              12/27/09
       http://bugs.python.org/issue7583    created  techtonik                     
                                                                               

datetime.rfcformat() for Date and Time on the Internet           12/27/09
       http://bugs.python.org/issue7584    created  techtonik                     
                                                                               

[patch] difflib should separate filename from timestamp with tab 12/27/09
       http://bugs.python.org/issue7585    created  techtonik                     
       patch                                                                   

Typo in collections documentation                                12/28/09
CLOSED http://bugs.python.org/issue7586    created  nneonneo                      
                                                                               

Python 3.1.1 source build issues                                 12/28/09
CLOSED http://bugs.python.org/issue7587    created  sharmaag                      
                                                                               

unittest.TestCase.shortDescription isn't short anymore           12/28/09
       http://bugs.python.org/issue7588    created  exarkun                       
                                                                               

setup.py shouldn't try to build nis module when nis headers aren 12/28/09
CLOSED http://bugs.python.org/issue7589    created  Arfrever                      
       patch                                                                   

'exceptions' module mentioned in documentation                   12/28/09
CLOSED http://bugs.python.org/issue7590    created  July                          
                                                                               

test_get_platform fails on 3.1                                   12/28/09
       http://bugs.python.org/issue7591    created  pitrou                        
       buildbot                                                                

ssl module documentation: SSLSocket.unwrap description shown twi 12/28/09
       http://bugs.python.org/issue7592    created  mnewman                       
                                                                               

Computed-goto patch for RE engine                                12/29/09
       http://bugs.python.org/issue7593    created  akuchling                     
       patch, needs review                                                     

shlex refactoring                                                12/29/09
       http://bugs.python.org/issue7594    created  ferringb                      
       patch, needs review                                                     

Doc typo for select.kevent()                                     12/29/09
CLOSED http://bugs.python.org/issue7595    created  whit537                       
                                                                               

test_logging sometimes fails                                     12/29/09
CLOSED http://bugs.python.org/issue7596    created  pitrou                        
                                                                               

curses.use_env implementation error                              12/30/09
       http://bugs.python.org/issue7597    created  kanru                         
       patch                                                                   

Cannot install, problems with assembly?                          12/30/09
CLOSED http://bugs.python.org/issue7598    created  sh.00                         
                                                                               

(c)ElementTree can produce invalid XML                           12/30/09
       http://bugs.python.org/issue7599    created  jwilk                         
                                                                               

interpreter crashes before start                                 12/30/09
CLOSED http://bugs.python.org/issue7600    created  mhh91                         
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc         12/30/09
CLOSED http://bugs.python.org/issue7601    created  sadhak                        
                                                                               

Doc: make clean and make update do not delete or update Doc/tool 12/30/09
CLOSED http://bugs.python.org/issue7602    created  flox                          
                                                                               

There is no 'seq' type                                           12/30/09
CLOSED http://bugs.python.org/issue7603    created  tom66                         
                                                                               

delattr __slots__ inconsistancy                                  12/30/09
CLOSED http://bugs.python.org/issue7604    created  ferringb                      
                                                                               

test_cmd_line fails with non-ascii path                          12/30/09
       http://bugs.python.org/issue7605    created  pitrou                        
                                                                               

test_xmlrpc fails with non-ascii path                            12/30/09
       http://bugs.python.org/issue7606    created  pitrou                        
                                                                               

stringlib fastsearch could be improved on 64-bit builds          12/30/09
       http://bugs.python.org/issue7607    created  pitrou                        
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.             12/30/09
CLOSED http://bugs.python.org/issue7608    created  alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                   12/31/09
CLOSED http://bugs.python.org/issue7609    created  Arfrever                      
       patch                                                                   

Cannot use both read and readline method in same ZipExtFile obje 12/31/09
       http://bugs.python.org/issue7610    created  lucifer                       
                                                                               

shlex not posix compliant when parsing "foo#bar"                 12/31/09
       http://bugs.python.org/issue7611    created  jjdmol2                       
                                                                               

Fix "pass and object" typo in Library Reference / Built-in Types 12/31/09
CLOSED http://bugs.python.org/issue7612    created  ygale                         
       patch                                                                   

[cppcheck] found a regression : Invalid number of character ((). 12/31/09
CLOSED http://bugs.python.org/issue7613    created  ettl.martin                   
       patch                                                                   

Python 2.6.4 segfaults                                           12/31/09
CLOSED http://bugs.python.org/issue7614    created  ttsiod                        
                                                                               

unicode_escape codec does not escape quotes                      01/01/10
       http://bugs.python.org/issue7615    created  rhansen                       
       patch                                                                   

test_memoryview test_setitem_writable failures with Intel ICC    01/01/10
       http://bugs.python.org/issue7616    created  ivank                         
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option 01/01/10
       http://bugs.python.org/issue7617    created  Arfrever                      
       patch                                                                   



Issues Now Closed (46)
______________________

True division of integers could be more accurate                  715 days
       http://bugs.python.org/issue1811    mark.dickinson                
       patch                                                                   

API to clear most free lists                                      690 days
       http://bugs.python.org/issue2019    amaury.forgeotdarc            
       patch                                                                   

unit test UnicodeWarning                                          687 days
       http://bugs.python.org/issue2100    ezio.melotti                  
                                                                               

test_disutils fails                                               568 days
       http://bugs.python.org/issue3054    ronaldoussoren                
       patch                                                                   

test_formatdate_usegmt fails on non-englisch locale               451 days
       http://bugs.python.org/issue4046    r.david.murray                
                                                                               

"??" in u"foo" raises a misleading error                          410 days
       http://bugs.python.org/issue4328    r.david.murray                
                                                                               

zipfile - add __exit__ attribute to make ZipFile object compatib  287 days
       http://bugs.python.org/issue5511    ezio.melotti                  
       patch                                                                   

test_urllib2 fails - urlopen error file not on local host         271 days
       http://bugs.python.org/issue5625    orsenthil                     
                                                                               

Improve --with-dbmliborder option                                 170 days
       http://bugs.python.org/issue6491    benjamin.peterson             
       patch                                                                   

Move the special-case for integer objects out of PyBytes_FromObj  141 days
       http://bugs.python.org/issue6687    alexandre.vassalotti          
       patch, 26backport                                                       

setup.py fails to find headers of system libffi                   104 days
       http://bugs.python.org/issue6943    benjamin.peterson             
       patch, easy                                                             

The replacement suggested for callable(x) in py3k is not	equival   92 days
       http://bugs.python.org/issue7006    benjamin.peterson             
       patch                                                                   

C/API - Document exceptions                                        89 days
       http://bugs.python.org/issue7033    lekma                         
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi   35 days
       http://bugs.python.org/issue7381    georg.brandl                  
       patch                                                                   

optparse Documentation References Example Files that do not Exis   30 days
       http://bugs.python.org/issue7404    georg.brandl                  
       patch                                                                   

datetime.datetime.isoformat truncation problem                     29 days
       http://bugs.python.org/issue7413    amaury.forgeotdarc            
       patch, needs review                                                     

doc: Code is not always colored                                     0 days
       http://bugs.python.org/issue7487    georg.brandl                  
                                                                               

float('nan')**2 != nan. float('nan')**2 error 33 on windows        13 days
       http://bugs.python.org/issue7534    mark.dickinson                
       patch                                                                   

If a generator raises TypeError when being unpacked, an unrelate   10 days
       http://bugs.python.org/issue7548    amaury.forgeotdarc            
                                                                               

ctypes doc improvement: c_char_p                                    6 days
       http://bugs.python.org/issue7569    georg.brandl                  
                                                                               

Strange issue : cursor.commit() with sqlite                         5 days
       http://bugs.python.org/issue7572    ghaering                      
                                                                               

tes_math fails Mac OS X 10.4 due to OverflowError in test_mtestf    5 days
       http://bugs.python.org/issue7575    mark.dickinson                
       patch                                                                   

documention buglet for PyBuffer                                     2 days
       http://bugs.python.org/issue7577    georg.brandl                  
       easy                                                                    

incomplete doc of zlib                                              0 days
       http://bugs.python.org/issue7581    amaury.forgeotdarc            
                                                                               

Typo in collections documentation                                   0 days
       http://bugs.python.org/issue7586    georg.brandl                  
                                                                               

Python 3.1.1 source build issues                                    0 days
       http://bugs.python.org/issue7587    amaury.forgeotdarc            
                                                                               

setup.py shouldn't try to build nis module when nis headers aren    1 days
       http://bugs.python.org/issue7589    benjamin.peterson             
       patch                                                                   

'exceptions' module mentioned in documentation                      1 days
       http://bugs.python.org/issue7590    georg.brandl                  
                                                                               

Doc typo for select.kevent()                                        0 days
       http://bugs.python.org/issue7595    georg.brandl                  
                                                                               

test_logging sometimes fails                                        1 days
       http://bugs.python.org/issue7596    ezio.melotti                  
                                                                               

Cannot install, problems with assembly?                             0 days
       http://bugs.python.org/issue7598    ezio.melotti                  
                                                                               

interpreter crashes before start                                    0 days
       http://bugs.python.org/issue7600    r.david.murray                
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            1 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

Doc: make clean and make update do not delete or update Doc/tool    0 days
       http://bugs.python.org/issue7602    georg.brandl                  
                                                                               

There is no 'seq' type                                              0 days
       http://bugs.python.org/issue7603    benjamin.peterson             
                                                                               

delattr __slots__ inconsistancy                                     0 days
       http://bugs.python.org/issue7604    benjamin.peterson             
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.                0 days
       http://bugs.python.org/issue7608    alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                      1 days
       http://bugs.python.org/issue7609    benjamin.peterson             
       patch                                                                   

Fix "pass and object" typo in Library Reference / Built-in Types    0 days
       http://bugs.python.org/issue7612    ezio.melotti                  
       patch                                                                   

[cppcheck] found a regression : Invalid number of character (().    0 days
       http://bugs.python.org/issue7613    ezio.melotti                  
       patch                                                                   

Python 2.6.4 segfaults                                              0 days
       http://bugs.python.org/issue7614    mark.dickinson                
                                                                               

Fwd: Debian Bug#42318: urllib.py has problems with malformed pro 3436 days
       http://bugs.python.org/issue210849  shinnosuke                    
                                                                               

urllib / urllib2 should cache 301 redirections                   2425 days
       http://bugs.python.org/issue735515  pitrou                        
                                                                               

fast modular exponentiation                                      2084 days
       http://bugs.python.org/issue936813  mark.dickinson                
       patch                                                                   

bdist_deb - Debian packager                                       316 days
       http://bugs.python.org/issue1054967 astraw                        
       patch                                                                   

Carbon.Scrap.PutScrapFlavor                                       987 days
       http://bugs.python.org/issue1700507 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 11 Add Option to Bind to a Local IP Address in httplib.py           462 days
open    http://bugs.python.org/issue3972   

  8 fast modular exponentiation                                     2084 days
closed  http://bugs.python.org/issue936813 

  6 [patch] difflib should separate filename from timestamp with ta    5 days
open    http://bugs.python.org/issue7585   

  6 [patch] diff.py to use iso timestamp                               5 days
open    http://bugs.python.org/issue7582   

  6 Implement fastsearch algorithm for rfind/rindex                   24 days
open    http://bugs.python.org/issue7462   

  5 test_xmlrpc fails with non-ascii path                              2 days
open    http://bugs.python.org/issue7606   

  5 test_logging sometimes fails                                       1 days
closed  http://bugs.python.org/issue7596   

  5 Patch to add docstrings to msvcrt                                  6 days
open    http://bugs.python.org/issue7579   

  5 Distutils-based installer does not detect 64bit versions of Pyt  126 days
open    http://bugs.python.org/issue6792   

  5 _sha256 et al. encode to UTF-8 by default                         17 days
open    http://bugs.python.org/issue3745   




From stefan_ml at behnel.de  Sun Jan  3 09:11:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Jan 2010 09:11:16 +0100
Subject: [Python-Dev] Providing support files to assist 3.x extension
	authors
In-Reply-To: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
References: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
Message-ID: <hhpjf3$lk1$1@ger.gmane.org>

Case Vanhorsen, 20.12.2009 01:38:
> When I ported gmpy (Python to GMP multiple precision library) to
> Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I
> found the code to slightly faster and cleaner than using PyLong_AsLong
> and checking for overflow.

You might want to look at the code Cython generates for integer type 
conversions. We use specialised coercion code that gets generated 
on-the-fly to convert Python long/int from and to all sorts of C integer 
types with compile time (portability) and runtime size/value checks. 
Depending on your needs, this may or may not be faster than the above C-API 
function.

Stefan



From david.lyon at preisshare.net  Mon Jan  4 00:42:15 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 10:42:15 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
Message-ID: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>


> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>
> This new operator removes the ambiguity the original proposal had,
> without making it more
> complex for common use cases. So if you dislike it, you will need to
> propose something
> else that also fixes the ambiguity we had.

Ok.

> Environment markers
>..
>Here are some example of fields using such markers:
>
>Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'

  Requires-Dist: [Windows] pywin32 1.0+

That's simpler, shorter, and less ambiguous. Easier to
parse for package managers.

David





From python at mrabarnett.plus.com  Mon Jan  4 01:14:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 04 Jan 2010 00:14:43 +0000
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4132F3.7020905@mrabarnett.plus.com>

David Lyon wrote:
>> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>>
>> This new operator removes the ambiguity the original proposal had,
>> without making it more
>> complex for common use cases. So if you dislike it, you will need to
>> propose something
>> else that also fixes the ambiguity we had.
> 
> Ok.
> 
>> Environment markers
>> ..
>> Here are some example of fields using such markers:
>>
>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.
> 
'win32' is more specific than 'Windows' and, to me, '1.0+' means
'>=1.0', not '>1.0'.


From martin at v.loewis.de  Mon Jan  4 01:21:53 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 04 Jan 2010 01:21:53 +0100
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4134A1.5090203@v.loewis.de>

>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.

Don't you want the PEP to complete? Why this bike-shedding?

I can agree it's shorter. I can't agree that it's simpler,
or less ambiguous.

Regards,
Martin


From ssteinerx at gmail.com  Mon Jan  4 01:29:14 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 3 Jan 2010 19:29:14 -0500
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
	Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <DF433474-9F8E-40BA-B0B1-D3021CAC6317@gmail.com>


On Jan 3, 2010, at 7:21 PM, Martin v. L?wis wrote:

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>> 
>>  Requires-Dist: [Windows] pywin32 1.0+
>> 
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
> 
> Don't you want the PEP to complete? Why this bike-shedding?

Really....

Enough, already!

S



From david.lyon at preisshare.net  Mon Jan  4 01:47:56 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 11:47:56 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <1349.218.214.45.58.1262566076.squirrel@syd-srv02.ezyreg.com>


Hi Martin, Happy New Year,

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>>
>>   Requires-Dist: [Windows] pywin32 1.0+
>>
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
>
> Don't you want the PEP to complete? Why this bike-shedding?

Well, I'm just helping out by pointing out some simpler ways
as Tarek asked me. I was only answering his question. I was out
celebrating so it took longer to reply than normal.

Bike-Shedding ? Me ? which bikeshed? wanting simple?

Anyway, I'm just reading the PEPs and commenting. If there
are some alterations that can be done, lets discuss them.

> I can agree it's shorter.
> ..

Cool.

What I'd really like is a 'Code-Repository:' keyword
so that we can install programs/libraries directly into
a system.

I feel that this would really simplify the packaging
landscape, making it much easier for the scientific
community and others to do python software installs.

This would allow us to perphaps have something even
*more modern* than CPAN.

So yes, getting PEP-345 right is important to me.

Have a nice day.

David





From abpillai at gmail.com  Mon Jan  4 10:08:05 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Mon, 4 Jan 2010 14:38:05 +0530
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <8548c5f31001040108v635a78ech33521371c6c50e82@mail.gmail.com>

On Sun, Dec 27, 2009 at 11:21 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Tue, Dec 15, 2009 at 12:37 AM, Steven Bethard
> <steven.bethard at gmail.com> wrote:
> >
> > If you're only concerned about 2.X, then yes, optparse will *never* be
> > removed from 2.X. There will be a deprecation note in the 2.X
> > documentation but deprecation warnings will only be issued when the -3
> > flag is specified. Please see the "Deprecation of optparse" section of
> > the PEP:
> >
> > http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
>
 Ref http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse .

 Considering that optparse will be deprecated like 5 years from now.
 I think this point is moot. The deprecation strategy IMHO is
 perhaps way too conservative. Maybe it should be deprecated
 faster ;)




-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100104/e0cabd8c/attachment-0004.html>

From fetchinson at googlemail.com  Mon Jan  4 10:32:10 2010
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 4 Jan 2010 10:32:10 +0100
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <fbe2e2101001040132o22302d2ct72b705ec046bcc46@mail.gmail.com>

>> If you're only concerned about 2.X, then yes, optparse will *never* be
>> removed from 2.X. There will be a deprecation note in the 2.X
>> documentation but deprecation warnings will only be issued when the -3
>> flag is specified. Please see the "Deprecation of optparse" section of
>> the PEP:
>>
>> http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
> Compare:
> http://docs.python.org/library/optparse.html
> http://argparse.googlecode.com/svn/tags/r101/doc/index.html
>
> argparse should be recommended as advanced and more featured variant
> of optparse - that goes without saying, but forcing people to switch
> and annoying them with deprecation messages brings only headache.

As has been noted already nobody is forcing people to switch. Optparse
will be available as a separate package and everybody will be free to
install it and will not have any deprecation messages anywhere.

> Just
> like optparse is better getopt, the latter could also be useful for
> people coming from other languages and porting their libraries to
> Python.
>
> I would better concentrate on real code examples how argparse solves
> problems and would really want to see
> http://www.python.org/dev/peps/pep-0389/#out-of-scope-various-feature-requests
> implemented until argparse enters phase where backward incompatible
> changes in API are impossible.
>
> I would prefer to see PEP 389 as a document describing proposed
> solutions to argument parsing problems rather than petition to replace
> one library with another. So, it should display common argument
> parsing scenarios (use cases) with examples that are also useful
> recipes for documentation. I guess more than 90% people here doesn't
> have time to read argparse methods descriptions to see what they could
> be useful for.



-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From olemis at gmail.com  Mon Jan  4 15:24:05 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 4 Jan 2010 09:24:05 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
Message-ID: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>

On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
> Thanks for the quick response.
>
> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> When I made that change, I didn't know that the __unittest "hack" was
>> being used elsewhere outside of unittest, so I felt fine replacing it
>> with another. While I still consider it an implementation detail, I
>> would be ok with exposing an "official" API for this. Perhaps
>> __unittest_ignore_traceback?
>
> Well, bazaar has had the trick for a couple of years, and googling
> around now turns up some other projects using it or thinking about it:
> <http://github.com/gfxmonk/mocktest/commit/b5e94f7ee06ab627cea2c9cf90d0aeb63ee5a698>
> <http://bitbucket.org/uche/amara/changeset/eeaf69f48271/>
> <http://twistedmatrix.com/trac/ticket/4127>
>

Add `dutest` and probably `nose` [2]_ and ...

> Reinstating the old implementation (with the same name) would mean
> that existing code would keep working with Python 2.7

IOW ... if it is removed all the libraries will have to change and
check for the Py version and ... (probably not a big deal depending on
the details of the ?solution?)

> but maybe a
> discussion could start about a new, less hacky, way of doing the same
>

I am strongly -1 for modifying the classes in ?traditional? unittest
module [2]_ (except that I am strongly +1 for the package structure
WITHOUT TOUCHING anything else ...) , and the more I think about it I
am more convinced ... but anyway, this not a big deal (and in the end
what I think is not that relevant either ... :o). So ...

pass

> May not be worthwhile making life more complicated though,
> there aren't *that* many unittest-extending projects.
>

But every library extending `unittest` will do it or otherwise
not-so-useful error messages will be displayed
;o)

PS: Happy New Year ! BTW

.. [1] [feature] nosexml should omit trace frames where `__unittest...
        (http://code.google.com/p/python-nosexml/issues/detail?id=4#c0)

.. [2] Assessment of unittest 2.7 API : new features and opinions
        (http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free new ripit 1.3.3 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/KFwyUTKH0vI/


From juanfhj at gmail.com  Tue Jan  5 17:10:16 2010
From: juanfhj at gmail.com (Juan Fernando Herrera J.)
Date: Tue, 5 Jan 2010 11:10:16 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards compatibility
Message-ID: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>

How about a new python 3 release with (possibly partial) backwards
compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
software hasn't been ported to it. I'm eager to use 3, but paradoxically,
the 3 release makes me rather stuck with 2.6. Excuse me if this has been
suggested in the past.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/7839cf7b/attachment-0006.htm>

From fuzzyman at voidspace.org.uk  Tue Jan  5 17:50:15 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 05 Jan 2010 16:50:15 +0000
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <4B436DC7.8040605@voidspace.org.uk>

On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
> How about a new python 3 release with (possibly partial) backwards 
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way 
> major software hasn't been ported to it. I'm eager to use 3, but 
> paradoxically, the 3 release makes me rather stuck with 2.6. Excuse me 
> if this has been suggested in the past.
>    

I don't know about other developers, but I certainly expected Python 3 
adoption to take a few years due to libraries only gradually making the 
jump. I also expected there to be nervousness and pain around the 
migration as well.

I think in fact that libraries *are* migrating and there are lots of 
encouraging signs. As a community we need to do all we can to promote 
Python 3 adoption, but I don't think there is much reason to be worried 
(or complacent for that matter).

All the best,

Michael Foord

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/d7b6baac/attachment-0006.htm>

From brian.curtin at gmail.com  Tue Jan  5 18:21:31 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jan 2010 11:21:31 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>

On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:

> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
The proper route to take, in my opinion, is to see what 2.x libraries you
are using that are not 3.x compatible, run 2to3 on them, then run their test
suite, and see where you get. Submit a patch or two to the library and see
what happens -- it at least gets the wheels in motion.

I'm sure everyone out there would like to dive into supporting 3.x, but it's
tough to prioritize when you are up to your eyeballs with 2.x bugs in your
tracker. Look at Twisted (
http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3)
-- over 1000 issues, ~5 developers -- 3.x support won't be here tomorrow,
but it's on the horizon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/67a4e6e2/attachment-0006.htm>

From guido at python.org  Tue Jan  5 18:23:08 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jan 2010 09:23:08 -0800
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B436DC7.8040605@voidspace.org.uk>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<4B436DC7.8040605@voidspace.org.uk>
Message-ID: <ca471dc21001050923i2ca37f6ej543faf0981c43b13@mail.gmail.com>

On Tue, Jan 5, 2010 at 8:50 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
>
> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
> I don't know about other developers, but I certainly expected Python 3
> adoption to take a few years due to libraries only gradually making the
> jump. I also expected there to be nervousness and pain around the migration
> as well.
>
> I think in fact that libraries *are* migrating and there are lots of
> encouraging signs. As a community we need to do all we can to promote Python
> 3 adoption, but I don't think there is much reason to be worried (or
> complacent for that matter).
>

Michael is right, but doesn't answer the OP's implied question "why can't
3.x be made backwards compatible with 2.6?"

Unfortunately the answer isn't easy. I wish it was "because we didn't have
time to do it" (since then the solution would be simply to call for more
volunteers to help out) -- but unfortunately, it comes closer to "we have no
idea how to do it."

Py3k was designed to be backwards incompatible, because that gives the most
long-term benefit of the language improvements. (Perhaps a better way of
saying this is that in the design of language improvements,
feature-by-feature backwards compatibility was intentionally not a
requirement, even though keeping most of the language mostly unchanged *was*
a requirement.)

In principle it would be possible to be more backwards compatible at the
syntactic level, but that's not where the pain of porting code lies -- the
major hurdles are typically he new way of thinking about Unicode (bytes vs.
text strings, instead of 8-bit-strings vs. Unicode strings), and the changed
semantics of dictionary keys and related APIs. Since these issues typically
exist across modules (passing strings and dicts between modules is common),
recognizing individual modules as "2.x" vs. "3.x" isn't possible.

Note, by the way, that you won't be "stuck" on 2.6 forever -- 2.7 alpha 1
was already released.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/eaba2f9b/attachment-0006.htm>

From regebro at gmail.com  Tue Jan  5 18:52:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 18:52:20 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <319e029f1001050952o71cb29afh66445550beeb99c2@mail.gmail.com>

On Tue, Jan 5, 2010 at 17:10, Juan Fernando Herrera J.
<juanfhj at gmail.com> wrote:
> I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.

Yes. Python 3 is not what you want to use today if you write
applications. If you write libraries 2010 is the year to port, IMO.
With some luck, 2011 will be the year to start porting applications
properly. We'll see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ianb at colorstudy.com  Tue Jan  5 20:00:49 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 13:00:49 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
Message-ID: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>

On Tue, Jan 5, 2010 at 11:21 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:
>
>> How about a new python 3 release with (possibly partial) backwards
>> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
>> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
>> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
>> suggested in the past.
>>
>>
> The proper route to take, in my opinion, is to see what 2.x libraries you
> are using that are not 3.x compatible, run 2to3 on them, then run their test
> suite, and see where you get. Submit a patch or two to the library and see
> what happens -- it at least gets the wheels in motion.
>

It's not even that easy -- libraries can't apply patches for Python 3
compatibility as they usually break Python 2 compatibility.  Potentially
libraries could apply patches that make a codebase 2to3 ready, but from what
I've seen that's more black magic than straight forward updating, as such
patches have to trick 2to3 producing the output that is desired.

The only workable workflow I've seen people propose for maintaining a single
codebase with compatibility across both 2 and 3 is to use such tricks, with
aliases to suppress some 2to3 updates when they are inappropriate, so that
you can run 2to3 on install and have a single canonical Python 2 source.
 Python 2.7 won't help much (even though it is trying) as the introduction
of non-ambiguous constructions like b"" aren't compatible with previous
versions of Python and so can't be used in many libraries (support at least
back to Python 2.5 is the norm for most libraries, I think).

Also, running 2to3 on installation is kind of annoying, as you get source
that isn't itself the canonical source, so to fix bugs you have to look at
the installed source and trace it back to the bug in the original source.

I suspect a reasonable workflow might be possible with hg and maybe patch
queues, but I don't feel familiar enough with those tools to map that out.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/e43ccd78/attachment-0006.htm>

From glyph at twistedmatrix.com  Tue Jan  5 21:24:45 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 5 Jan 2010 15:24:45 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.

From regebro at gmail.com  Tue Jan  5 21:57:35 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 21:57:35 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <319e029f1001051257k3827c52ahcd115b15d9a8ece7@mail.gmail.com>

On Tue, Jan 5, 2010 at 21:24, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It seems like this is a problem to be addressed, then. ?Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

Some of it is about changing the code so 2to3 doesn't have to do ugly
things. For example, always use iteritems(), so that 2to3 knows that
items() can be used without converting it to a list, etc. Then we do
have the problems with unicode vs string vs bytes, where I can't think
of a clever solution except your no-op shims, which admittedly is
fugly . For me that tends to turn into testing everything until the
tests run on all platforms, which I guess is close to "black magic".
Don't know what to do about that.

python-incompatibility is about documenting all differences, and also
how to make code that runs under both 2.6 and 3.0 without 2to3. But I
guess it should be extended into how to spell something that is clean
in both 2.6 and 3.x.

>> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.

Ian: If you have examples of 2to3 updated that are not appropriate
(except the x.items() -> list(x.items()) they would be appreciated.

> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

In my experience it turned out to be less of a problem than I thought.
That is to some extent because the modules I've ported has had good
test coverage, and I have fixed 99.9% of the bugs by making the tests
pass. Developing with Distributes 2to3 support has then been quite
smooth and in most cases the separation between the 2.x source and the
3.x source hasn't been a problem.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan  5 22:07:23 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 22:07:23 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <4B43AA0B.5030308@v.loewis.de>

> It's not even that easy -- libraries can't apply patches for Python 3
> compatibility as they usually break Python 2 compatibility.  Potentially
> libraries could apply patches that make a codebase 2to3 ready, but from
> what I've seen that's more black magic than straight forward updating,
> as such patches have to trick 2to3 producing the output that is desired.

I wouldn't qualify it in that way. It may be necessary occasionally to
trick 2to3, but that's really a bug in 2to3 which you should report, so
that trickery is then a work-around for a bug - something that you may
have to do with other API, as well.

The "black magic" is really more in the parts that 2to3 doesn't touch
at all (because they are inherently not syntactic); these are the
problem areas Guido refers to. The "black magic" then is to make the
same code work unmodified for both 2.x and 3.x.

> The only workable workflow I've seen people propose for maintaining a
> single codebase with compatibility across both 2 and 3 is to use such
> tricks, with aliases to suppress some 2to3 updates when they are
> inappropriate

I think you misunderstand. All this is necessary, but *not* to suppress
2to3 updates. More typically, it is necessary because 2to3 leaves the
code unmodified either way.

> Also, running 2to3 on installation is kind of annoying, as you get
> source that isn't itself the canonical source, so to fix bugs you have
> to look at the installed source and trace it back to the bug in the
> original source.

That's an issue indeed, but one that I would hope that can be fixed by
improved traceback printing. It would be good if such traceback printing
could make it into 2.7.

Regards,
Martin


From martin at v.loewis.de  Tue Jan  5 22:13:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Jan 2010 22:13:07 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <4B43AB63.3000607@v.loewis.de>

> No-op constructions like 'bytes("")' could help for older versions of
> Python, though.  A very, very small runtime shim could provide
> support for these, if 2to3 could be told about it somehow.

You actually don't *need* 2to3 support for that - bytes("") can work
in either version:

2.x:
def bytes(s):
  return s

3.x:
def bytes(s)
  return s.encode("ascii")

On top of that, you *could* as for bytes("") to be converted to b"" in
3.x - and it is indeed possible to tell 2to3 about that, and you don't
even need to modify 2to3's source to do so: it can be part of your
package.

>> Also, running 2to3 on installation is kind of annoying, as you get
>> source that isn't itself the canonical source, so to fix bugs you
>> have to look at the installed source and trace it back to the bug
>> in the original source.
> 
> Given the way tracebacks are built, i.e. from filenames stored in
> .pycs rather than based on where the code was actually loaded in the
> filesystem, couldn't 2to3 could do .pyc rewriting to point at the
> original source?  Sort of like our own version of the #line
> directive? :)

I think it could, but it would be fairly flaky as the pycs
can get lost, and lose that information after regeneration. Also,
it may be confusing in other scenarios.

I'd rather have it generate separate mapper files, and change the
traceback printing to consider these (as an option).

> Seriously though, I find it hard to believe that this is a big
> problem.  The 3.x source looks pretty similar to the 2.x source, and
> it's good to look at both if you're dealing with a 3.x issue.

That's my experience as well - the only challenge is that you can't
cut-n-paste the source URL into an editor.

Regards,
Martin


From ianb at colorstudy.com  Tue Jan  5 22:39:21 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 15:39:21 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B43AA0B.5030308@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com> 
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com> 
	<4B43AA0B.5030308@v.loewis.de>
Message-ID: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>

On Tue, Jan 5, 2010 at 3:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > It's not even that easy -- libraries can't apply patches for Python 3
> > compatibility as they usually break Python 2 compatibility. ?Potentially
> > libraries could apply patches that make a codebase 2to3 ready, but from
> > what I've seen that's more black magic than straight forward updating,
> > as such patches have to trick 2to3 producing the output that is desired.
>
> I wouldn't qualify it in that way. It may be necessary occasionally to
> trick 2to3, but that's really a bug in 2to3 which you should report, so
> that trickery is then a work-around for a bug - something that you may
> have to do with other API, as well.
>
> The "black magic" is really more in the parts that 2to3 doesn't touch
> at all (because they are inherently not syntactic); these are the
> problem areas Guido refers to. The "black magic" then is to make the
> same code work unmodified for both 2.x and 3.x.

Just to clarify, the black magic I'm referring to is things like:

try:
?? ?unicode_ = unicode
except NameError:
?? ?unicode_ = str

and some other aliases like this that are unambiguous and which 2to3
won't touch (if you write them correctly). ?If the porting guide noted
all these tricks (of which several have been developed, and I'm only
vaguely aware of a few) that would be helpful. ?It's not a lot of
tricks, but the tricks are not obvious and 2to3 gets the translation
wrong pretty often without them. ?For instance, when I say str in
Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
str and unicode to str. ?That *nothing* translates to bytes by default
(AFAIK) means that people must either be living in a bytes-free world
(which sure, lots of code does) or they are using tricks not included
in 2to3 itself.


Also replying to Glyph:
> > Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.
>
> Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in
the filesystem, couldn't 2to3 could do .pyc rewriting to point at the
original source? ?Sort of like our own version of the #line directive?
:)
>
> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
then I don't currently develop any code that is "installed", I only
develop code that is directly from a source code checkout, and where
the checkout is put on the path.  I guess I could have something that
automatically builds the code on every edit, and that's not
infeasible.  It's just not fun.  So long as I have to support Python 2
(which is like forever) then adding Python 3 only makes development
that much more complicated and much less fun, with no concrete
benefits.  Which is a terribly crotchety of me.  Sorry.

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker


From martin at v.loewis.de  Tue Jan  5 23:23:53 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 23:23:53 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
Message-ID: <4B43BBF9.2000302@v.loewis.de>

> Just to clarify, the black magic I'm referring to is things like:
> 
> try:
>     unicode_ = unicode
> except NameError:
>     unicode_ = str
> 
> and some other aliases like this that are unambiguous and which 2to3
> won't touch (if you write them correctly).  If the porting guide noted
> all these tricks (of which several have been developed, and I'm only
> vaguely aware of a few) that would be helpful.  It's not a lot of
> tricks, but the tricks are not obvious and 2to3 gets the translation
> wrong pretty often without them.  For instance, when I say str in
> Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
> str and unicode to str.

No, that's not what's happening. Instead, str is not translated at all
(i.e. it's *not* translated to str - it just isn't touched).

Translating unicode to str is always correct, AFAICT.

I'm not quite sure what the magic above is supposed to achieve: in 2.x,
unicode_ becomes an alias to unicode, in 3.x, 2to3 translates unicode
to str, and then the block becomes

try:
  unicode_ = str
except NameError:
  unicode_ = str

so the NameError is actually never triggered.

Could it be that the magic is proposed for a mode where you *don't*
use 2to3?

> That *nothing* translates to bytes by default
> (AFAIK) means that people must either be living in a bytes-free world
> (which sure, lots of code does) or they are using tricks not included
> in 2to3 itself.

By your above definition of "translates", the function "bytes" is
translated to bytes (i.e. it isn't touched by 2to3).

> Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
> then I don't currently develop any code that is "installed", I only
> develop code that is directly from a source code checkout, and where
> the checkout is put on the path.  I guess I could have something that
> automatically builds the code on every edit, and that's not
> infeasible.  It's just not fun.

Depends on where you draw your fun from. It is indeed fun to me, but
I can see why it may not be fun to you. So you could ask me to do it for
you - or you may try to use what I have already done for you, so you
don't have to do it.

> So long as I have to support Python 2
> (which is like forever) then adding Python 3 only makes development
> that much more complicated and much less fun, with no concrete
> benefits.

I doubt it will be *much* more complicated - only a little.
As for concrete benefits: there may be no benefits at this point,
but in the long run, starting now will have saved you a lot of
pressure in the long run, and stop users from switching away from
your packages because of lack of Python 3 support.

Regards,
Martin


From ziade.tarek at gmail.com  Wed Jan  6 01:08:34 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 01:08:34 +0100
Subject: [Python-Dev] PEP 386 and PEP 345
Message-ID: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>

Hi,

I think we've reached a consensus on those two PEPs.

Although, there's one last point that was forgotten in the discussions
: I've introduced "rc" in the pre-releases markers, so PEP 386 is
compatible with Python's own version scheme.  "rc" comes right after
"c" in the sorting. It's slightly redundant with the "c" marker but I
don't think this really matters as long as consumers know how to order
them (a < b < c < rc). I have also stated that "c" is the preferred
marker for third party projects, from PEP 386 point of view.

Is there anything else I can do to make those two PEPs accepted ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From david.lyon at preisshare.net  Wed Jan  6 01:26:34 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:26:34 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <4B43BBF9.2000302@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
	<4B43BBF9.2000302@v.loewis.de>
Message-ID: <1322.218.214.45.58.1262737594.squirrel@syd-srv02.ezyreg.com>

Hi Martin,

> ... but in the long run, starting now will have saved you a lot of
> pressure in the long run, and stop users from switching away from
> your packages because of lack of Python 3 support.

In a production situation it works the other way around. If there's
an application that requires twisted (or whatever package) then most
people would use the appropriate interpreter to match the library.

Since you guys all did your jobs so well :-) doing so is painless.

Because there is so much "comfort" with the existing situation it
makes it an effort for people to move to a different lounge chair.
Namely python 3.

I'd suggest that moving the package set (pypi) to python 3 could
be kicked along with the help of some automated tools. I don't
know what tools you guys have got.

But I am very sure that if code analysis was provided to package
developers on python 3 (so they don't have to run it themselves),
then it would be like an even bigger tv screen in a bigger lounge
room and it would assist in drawing them over.

David







From david.lyon at preisshare.net  Wed Jan  6 01:36:24 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:36:24 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <1337.218.214.45.58.1262738184.squirrel@syd-srv02.ezyreg.com>


Hi Ian,

> The only workable workflow I've seen people propose for maintaining a
> single
> codebase with compatibility across both 2 and 3 is to use such tricks,
> with
> aliases to suppress some 2to3 updates when they are inappropriate, so that
> you can run 2to3 on install and have a single canonical Python 2 source.
>  Python 2.7 won't help much (even though it is trying) as the introduction
> of non-ambiguous constructions like b"" aren't compatible with previous
> versions of Python and so can't be used in many libraries (support at
> least
> back to Python 2.5 is the norm for most libraries, I think).
>
> Also, running 2to3 on installation is kind of annoying, as you get source
> that isn't itself the canonical source, so to fix bugs you have to look at
> the installed source and trace it back to the bug in the original source.
>
> I suspect a reasonable workflow might be possible with hg and maybe patch
> queues, but I don't feel familiar enough with those tools to map that out.

That's why I am saying that we need a Code-Repository: section in PEP-345
Metadata.

Let package developers keep two branches in SCM. One for 2.x and one for 3.x

Let them backport features from their 3.x development series to their 2.x
code base. In exactly the same way that it is done in so many other
developments today.

Keeping multiple branches of code is a very common thing these days. And
there are tools in the outside world (bzr,svn,mercurial) that allow us to
do it very easily.

Let's have that support in python; it will make life easier.

David










From david.lyon at preisshare.net  Wed Jan  6 03:01:22 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 13:01:22 +1100 (EST)
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <1617.218.214.45.58.1262743282.squirrel@syd-srv02.ezyreg.com>

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?

Tarek,

Given that I helped out so much last year with the PEP in discussing
different options, even if they weren't accepted, I really feel that it is
unfair if my name isn't mentioned. It was a huge time sacrifice on my
part.

For example, even if I only managed to explain the version numbering and
clarify how that worked. It did take me some time to do that.

What I did do however, was spend a lot of time with the multiplatform
"Markers". I still think that two short weeks more of "discussion" could
resolve some issues. That discussion went for 4 months on distutils-ml.

Look, major issues aside, can you make the following concessions on
PEP-345 which I only feel will help it, namely:

 1) Source-Repository: specify a code repository to install from

 2) Streamline Requires-Python: by implementing "Markers" as
    noted by the PEP. A marker being something like
    "Requires-Python(windows): lxml". Otherwise remove the
    word marker from the PEP and just replace with "metacode".
    Markers are what were discussed on distutils-ml. Metacode
    is what is described in the PEP.

 3) Remove the inconsistency and platform ambiguities. Especially
    for windows users. For example, "win32" is extremely confusing
    for windows users right now. As more and more systems now are
    64 bit. Use the platform module, instead of the sys module
    for constants. I'll post to distutils-ml on this.

I am certainly not trying to hold this PEP up, and I apologise on
my part for my late attention. I will post to distutils-ml on these
and i promise to keep my comments unheated and unwitty.

Having said that, PEP-345 is *super-important*. A week or two or three
more discussion and the issues can be resolved.

We all just want to focus on being productive. It would be a great
accomplishment for you to get PEP-345 approved and likewise for
me getting mentioned even in a minor role as helping out on a PEP.

So I'm hoping that you can make a few last minute concessions
meaning that we can all happily go on our way in 2010.

Regards

David













From brett at python.org  Wed Jan  6 06:20:30 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Jan 2010 21:20:30 -0800
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <bbaeab101001052120qe888df8o7eb03a61e6c030c7@mail.gmail.com>

On Tue, Jan 5, 2010 at 16:08, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?
>

As you said, consensus has been reached, so just Guido's BDFL stamp of
approval is all I can think of.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/a76cb75b/attachment-0006.htm>

From chris at simplistix.co.uk  Wed Jan  6 12:19:45 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:19:45 +0000
Subject: [Python-Dev] bug triage
Message-ID: <4B4471D1.9020707@simplistix.co.uk>

Hi All,

Is there a high volume of incoming bugs to the Python tracker?
If so, I'd like to help with triaging. I think I have all the necessary 
access, what I'm missing is the knowledge of how to set myself up to get 
notifications of new bugs...

How do I do that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From fuzzyman at voidspace.org.uk  Wed Jan  6 12:24:37 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 06 Jan 2010 11:24:37 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <4B4472F5.8000709@voidspace.org.uk>

On 06/01/2010 11:19, Chris Withers wrote:
> Hi All,
>
> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the 
> necessary access, what I'm missing is the knowledge of how to set 
> myself up to get notifications of new bugs...
>
> How do I do that?
>

Bug triaging is one of Python's "big needs" and anything you do to help 
on this score would be much appreciated. Particularly reviewing new and 
outstanding issues.

I assumed there would be RSS feeds for bug tracker activity but can't 
easily find these on the tracker. There is a bot that posts activity to 
#python-dev, so there must be some way of getting this information.

All the best,

Michael



> cheers,
>
> Chris
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From solipsis at pitrou.net  Wed Jan  6 12:25:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 11:25:16 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <loom.20100106T122258-896@post.gmane.org>


Hi Chris,

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary 
> access, what I'm missing is the knowledge of how to set myself up to get 
> notifications of new bugs...

Do you really want to get such notifications? There may be a lot of them.
If you want however, you can join #python-dev on IRC (irc.freenode.net) where
there's a bot which posts updates of all bugs on the tracker. There's usually
not a lot of discussion going on so you probably won't feel flooded.

In addition to bug triage, what is needed is reviewing of existing patches, as
well as writing patches for issues which haven't been addressed yet :-)

Regards

Antoine.




From chris at simplistix.co.uk  Wed Jan  6 12:30:40 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:30:40 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B447460.7080100@simplistix.co.uk>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't 
> easily find these on the tracker. There is a bot that posts activity to 
> #python-dev, so there must be some way of getting this information.

Yeah, email-out is what I'm really after... I have it for my own Roundup 
instance so it can't be that hard to do ;-)

Roch?, you guys host the bug tracker, right? Is there email-out set up 
for it?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ncoghlan at gmail.com  Wed Jan  6 12:37:23 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 21:37:23 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B4475F3.5040406@gmail.com>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't
> easily find these on the tracker. There is a bot that posts activity to
> #python-dev, so there must be some way of getting this information.

I'm pretty sure the bugs list is still the primary spooled notification
mechanism:
http://mail.python.org/mailman/listinfo/python-bugs-list

There are also the weekly tracker activity summaries that are posted
here to python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From chris at simplistix.co.uk  Wed Jan  6 12:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:41:28 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <4B4476E8.8050709@simplistix.co.uk>

Nick Coghlan wrote:
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

That's what I was after, thanks!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From facundobatista at gmail.com  Wed Jan  6 13:03:08 2010
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 6 Jan 2010 09:03:08 -0300
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <e04bdf311001060403y3b65c647jf82fdc26266a0efe@mail.gmail.com>

On Wed, Jan 6, 2010 at 8:19 AM, Chris Withers <chris at simplistix.co.uk> wrote:

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary
> access, what I'm missing is the knowledge of how to set myself up to get
> notifications of new bugs...

Not notifications, but maybe a way to have a higher look of them for
easy selection:

  http://www.taniquetil.com.ar/cgi-bin/pytickets.py

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ziade.tarek at gmail.com  Wed Jan  6 13:29:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:29:58 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>

On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 06/01/2010 11:19, Chris Withers wrote:
>>
>> Hi All,
>>
>> Is there a high volume of incoming bugs to the Python tracker?
>> If so, I'd like to help with triaging. I think I have all the necessary
>> access, what I'm missing is the knowledge of how to set myself up to get
>> notifications of new bugs...
>>
>> How do I do that?
>>
>
> Bug triaging is one of Python's "big needs" and anything you do to help on
> this score would be much appreciated. Particularly reviewing new and
> outstanding issues.

Another useful triage I think, is to review the oldest bugs (some of
them are > 5 years)
and remove the ones that are not relevant anymore, or duplicate with
newer entries.

Tarek

-- 
Tarek Ziad? | http://ziade.org


From chris at simplistix.co.uk  Wed Jan  6 13:31:08 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 12:31:08 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B44828C.4070201@simplistix.co.uk>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this 
with someone as a paired task for those 2 days...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Wed Jan  6 13:37:55 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:37:55 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B44828C.4070201@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B44828C.4070201@simplistix.co.uk>
Message-ID: <94bdd2611001060437s2d0242aembc669c3263773fea@mail.gmail.com>

On Wed, Jan 6, 2010 at 1:31 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Another useful triage I think, is to review the oldest bugs (some of
>> them are > 5 years)
>> and remove the ones that are not relevant anymore, or duplicate with
>> newer entries.
>
> I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this with
> someone as a paired task for those 2 days...

I'll be doing Distutils stuff but I can probably help around a bit in that task:
Distutils have quite a few old issues so I can tackle those


From roche at upfrontsystems.co.za  Wed Jan  6 13:32:39 2010
From: roche at upfrontsystems.co.za (=?ISO-8859-1?Q?Roch=E9?= Compaan)
Date: Wed, 06 Jan 2010 14:32:39 +0200
Subject: [Python-Dev] bug triage
In-Reply-To: <4B447460.7080100@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B447460.7080100@simplistix.co.uk>
Message-ID: <1262781159.2836.218.camel@didi>

On Wed, 2010-01-06 at 11:30 +0000, Chris Withers wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't 
> > easily find these on the tracker. There is a bot that posts activity to 
> > #python-dev, so there must be some way of getting this information.
> 
> Yeah, email-out is what I'm really after... I have it for my own Roundup 
> instance so it can't be that hard to do ;-)
> 
> Roch?, you guys host the bug tracker, right? Is there email-out set up 
> for it?

We do, but we don't administer it. There are a few administrators taking
care of it and you should be able to reach them by logging your request
here: http://psf.upfronthosting.co.za/roundup/meta/ or post it to the
Infrastructure mailing list: infrastructure at python.org


-- 
Roch? Compaan
Upfront Systems                   http://www.upfrontsystems.co.za



From ncoghlan at gmail.com  Wed Jan  6 13:57:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 22:57:24 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B4488B4.2070208@gmail.com>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I believe someone (Daniel Diniz, maybe?) did do a pass over those some
time in the  last 12 months, so most of the obviously irrelevant ones
that are that old should already be gone. Not to say it isn't worth
doing another pass, just saying not to get disheartened if there aren't
many that can be readily closed.

There are at least a few still kicking around just because they're
difficult to deal with (there's an ancient one to do with one of the
ways circular imports can fail that I occasionally go back and reread
before moving on to something more tractable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rdmurray at bitdance.com  Wed Jan  6 14:43:17 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 08:43:17 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4476E8.8050709@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
	<4B4476E8.8050709@simplistix.co.uk>
Message-ID: <20100106134317.3EF141FB5A6@kimball.webabinitio.net>

On Wed, 06 Jan 2010 11:41:28 +0000, Chris Withers <chris at simplistix.co.uk> wrote:
> Nick Coghlan wrote:
> > I'm pretty sure the bugs list is still the primary spooled notification
> > mechanism:
> > http://mail.python.org/mailman/listinfo/python-bugs-list
> 
> That's what I was after, thanks!

Just for completeness, there's also new-bugs-announce if you want
just *new* bug notification.  That's more for people who want to
watch for bugs they want to become nosy on, though; if you are
doing triage python-bugs-list is what you want.

Please also read http://www.python.org/dev/workflow/ if you haven't
already.  Thanks for being willing to chip in!

--David


From brian.curtin at gmail.com  Wed Jan  6 15:57:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 08:57:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4488B4.2070208@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
Message-ID: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I believe someone (Daniel Diniz, maybe?) did do a pass over those some
> time in the  last 12 months, so most of the obviously irrelevant ones
> that are that old should already be gone. Not to say it isn't worth
> doing another pass, just saying not to get disheartened if there aren't
> many that can be readily closed.
>
> There are at least a few still kicking around just because they're
> difficult to deal with (there's an ancient one to do with one of the
> ways circular imports can fail that I occasionally go back and reread
> before moving on to something more tractable).
>
> Cheers,
> Nick.
>

On the topic of bugs that can be readily closed (literally), I've recently
come across a number of issues which appear to be sitting in a patch or
review stage, but their patches have been committed and the issue remains
open. What is the best course of action there? I'd just go ahead and close
the issue myself but I don't have tracker privileges.

I'm willing to help out with another Daniel Diniz-esque triage sweep if that
would help.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/05d9ebd7/attachment-0006.htm>

From ssteinerx at gmail.com  Wed Jan  6 16:14:20 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 6 Jan 2010 10:14:20 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <7FCF8B19-3465-4392-80CC-BEAEBD926ECA@gmail.com>


On Jan 6, 2010, at 7:29 AM, Tarek Ziad? wrote:

> On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> On 06/01/2010 11:19, Chris Withers wrote:
>>> 
>>> Hi All,
>>> 
>>> Is there a high volume of incoming bugs to the Python tracker?
>>> If so, I'd like to help with triaging. I think I have all the necessary
>>> access, what I'm missing is the knowledge of how to set myself up to get
>>> notifications of new bugs...
>>> 
>>> How do I do that?
>>> 
>> 
>> Bug triaging is one of Python's "big needs" and anything you do to help on
>> this score would be much appreciated. Particularly reviewing new and
>> outstanding issues.
> 
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I was actually thinking about that the other day when I saw that the average age of bugs on the Python tracker was at some hideously large 3 digit number.  

The 'success' statistic would be to bring that down below, say, 100.

S



From skip at pobox.com  Wed Jan  6 17:38:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 6 Jan 2010 10:38:13 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <19268.48245.616046.874126@montanaro.dyndns.org>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> I'm pretty sure the bugs list is still the primary spooled
    Nick> notification mechanism:
    Nick> http://mail.python.org/mailman/listinfo/python-bugs-list

Actually, there is a new-bugs-announce list:

    http://mail.python.org/mailman/listinfo/new-bugs-announce

Skip


From solipsis at pitrou.net  Wed Jan  6 18:56:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 17:56:49 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <hi2it0$q48$1@ger.gmane.org>

Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> On the topic of bugs that can be readily closed (literally), I've
> recently come across a number of issues which appear to be sitting in a
> patch or review stage, but their patches have been committed and the
> issue remains open. What is the best course of action there?

Post a message on the issue asking for info.




From solipsis at pitrou.net  Wed Jan  6 19:09:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 18:09:39 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<hi2it0$q48$1@ger.gmane.org>
Message-ID: <loom.20100106T190650-337@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> > On the topic of bugs that can be readily closed (literally), I've
> > recently come across a number of issues which appear to be sitting in a
> > patch or review stage, but their patches have been committed and the
> > issue remains open. What is the best course of action there?
> 
> Post a message on the issue asking for info.

Ok, I realize my answer might have been a bit terse :-)
The patch might be waiting to be merged in all development branches, or it may
not totally resolve the issue, or perhaps documentation needs to be updated, or
perhaps it is pending a verdict from the buildbots, etc. You can't deduce that
the issue is completely fixed from the simple fact that something has been
committed.

Regards

Antoine Pitrou.






From brett at python.org  Wed Jan  6 20:03:32 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 11:03:32 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>  I believe someone (Daniel Diniz, maybe?) did do a pass over those some
>> time in the  last 12 months, so most of the obviously irrelevant ones
>> that are that old should already be gone. Not to say it isn't worth
>> doing another pass, just saying not to get disheartened if there aren't
>> many that can be readily closed.
>>
>> There are at least a few still kicking around just because they're
>> difficult to deal with (there's an ancient one to do with one of the
>> ways circular imports can fail that I occasionally go back and reread
>> before moving on to something more tractable).
>>
>> Cheers,
>> Nick.
>>
>
> On the topic of bugs that can be readily closed (literally), I've recently
> come across a number of issues which appear to be sitting in a patch or
> review stage, but their patches have been committed and the issue remains
> open. What is the best course of action there? I'd just go ahead and close
> the issue myself but I don't have tracker privileges.
>
>
If a core developer is willing to step forward and vouch for you to get
tracker privileges then I will give them to you. We are trying to give out
tracker privs w/ less time than required to get commit privileges. So as
long as you have helped out on a few issues in a positive and correct way
that should be enough to get one of the regulars who perform triage to
notice.

-Brett



I'm willing to help out with another Daniel Diniz-esque triage sweep if that
> would help.
>
> Brian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/f24c9595/attachment-0006.htm>

From nad at acm.org  Wed Jan  6 21:41:05 2010
From: nad at acm.org (Ned Deily)
Date: Wed, 06 Jan 2010 12:41:05 -0800
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <nad-19B167.12410506012010@news.gmane.org>

In article <4B4475F3.5040406 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't
> > easily find these on the tracker. There is a bot that posts activity to
> > #python-dev, so there must be some way of getting this information.
> 
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

Also, that mailing list (along with most python development related 
mailing lists) is mirrored at gmane.org which means it can also be 
obtained via a newsreader (NNTP) or various RSS feeds.

http://dir.gmane.org/gmane.comp.python.bugs

-- 
 Ned Deily,
 nad at acm.org



From nick.bastin at gmail.com  Wed Jan  6 22:14:54 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:14:54 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
Message-ID: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>

(This may occur on more platforms - I can test on more unix platforms
if the consensus is this is an actual problem and I'm not just a nut)

On freebsd5, if you do a simple ./configure --enable-shared in current
(2.7) trunk, your python shared library will build properly, but all
modules will fail to find the shared library and thus fail to build:

gcc -shared build/temp.freebsd-5.3-RELEASE-i386-2.7/u1/Python/Python-2.7a1/Modules/_struct.o
   -L/u1/tmp/python2.7a1/lib -L/usr/local/lib -lpython2.7 -o
build/lib.freebsd-5.3-RELEASE-i386-2.7/_struct.so
/usr/bin/ld: cannot find -lpython2.7
building '_ctypes_test' extension
...

This of course is because libpython2.7.so is in the current directory
and not (yet) installed in /usr/local/lib.  I've made a very simple
fix for this problem that works, but at least to me smells a bit
funny, which is to modify setup.py to add the following to
detect_modules():

        # If we did --enable-shared, we need to be able to find the library
        # we just built in order to build the modules.
        if platform == 'freebsd5':
            add_dir_to_list(self.compiler_obj.library_dirs, '.')


Which brings me to a few questions:

a) Does this seem like a real problem, or am I missing something obvious?

b) Does this fix seem like the sensible thing to do?  (it seems at
least that we ought to check that the user configured --enable-shared
and only set -L. in that case, if that's possible)

Setting --enable-shared when you actually have a libpython2.7.so in
/usr/local/lib (or whatever --prefix you've selected) is possibly even
more dangerous, because it may succeed in linking against a
differently-built library than what you intended.

--
Nick


From nick.bastin at gmail.com  Wed Jan  6 22:39:17 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:39:17 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <66d0a6e11001061339t38202b70mca1091e1926ecf4b@mail.gmail.com>

On Wed, Jan 6, 2010 at 16:14, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> This of course is because libpython2.7.so is in the current directory
> and not (yet) installed in /usr/local/lib.

One minor correction - as you could see from the compile line, the
actual --prefix in this case is /u1/tmp/python2.7a1, but the libraries
obviously aren't installed there yet either.  Perhaps a better fix
than setting -L. would be to put the shared library in
build/lib.freebsd-5.3-RELEASE-i386-2.7 and add that to the library
path for the linker (the build creates this directory, but installs
nothing in it).

--
Nick


From martin at v.loewis.de  Wed Jan  6 23:21:44 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Jan 2010 23:21:44 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <4B450CF8.8090009@v.loewis.de>

> b) Does this fix seem like the sensible thing to do?

No. Linking in setup.py should use the same options as if the module
was built as *shared* through Modules/Setup, which, IIUC, should use
BLDLIBRARY.

Regards,
Martin


From nick.bastin at gmail.com  Thu Jan  7 01:08:13 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 19:08:13 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B450CF8.8090009@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
Message-ID: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:21, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> b) Does this fix seem like the sensible thing to do?
>
> No. Linking in setup.py should use the same options as if the module
> was built as *shared* through Modules/Setup, which, IIUC, should use
> BLDLIBRARY.

Thanks for that pointer, that makes much more sense.  Indeed,
BLDLIBRARY on FreeBSD* is set to '-L. -lpython$(VERSION)' if you set
--enable-shared, but somehow that piece of information doesn't
propagate into the module build.  More investigation to be done...

--
Nick


From rdmurray at bitdance.com  Thu Jan  7 02:22:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 20:22:42 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
Message-ID: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>


On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On the topic of bugs that can be readily closed (literally), I've recently
> > come across a number of issues which appear to be sitting in a patch or
> > review stage, but their patches have been committed and the issue remains
> > open. What is the best course of action there? I'd just go ahead and close
> > the issue myself but I don't have tracker privileges.
> >
> >
> If a core developer is willing to step forward and vouch for you to get
> tracker privileges then I will give them to you. We are trying to give out
> tracker privs w/ less time than required to get commit privileges. So as
> long as you have helped out on a few issues in a positive and correct way
> that should be enough to get one of the regulars who perform triage to
> notice.
> 
> -Brett

I've done a quick scan of issues Brian is nosy on to refresh my
memory, and I'd say he's definitely been making positive contributions.
I'm willing to volunteer to keep an eye on his triage work for a while
if you grant him tracker privs.

Brian, I assume you'll be cognizant of Antoine's advice about making
sure a bug really should be closed before closing it :)  Hanging out in
#python-dev on freenode while working on issues can be helpful, as well,
since you can quickly ask whoever is there for second opinions on
particular bugs.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Thu Jan  7 02:28:26 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 17:28:26 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com> 
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com> 
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
Message-ID: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com> wrote:

>
> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On the topic of bugs that can be readily closed (literally), I've
> recently
> > > come across a number of issues which appear to be sitting in a patch or
> > > review stage, but their patches have been committed and the issue
> remains
> > > open. What is the best course of action there? I'd just go ahead and
> close
> > > the issue myself but I don't have tracker privileges.
> > >
> > >
> > If a core developer is willing to step forward and vouch for you to get
> > tracker privileges then I will give them to you. We are trying to give
> out
> > tracker privs w/ less time than required to get commit privileges. So as
> > long as you have helped out on a few issues in a positive and correct way
> > that should be enough to get one of the regulars who perform triage to
> > notice.
> >
> > -Brett
>
> I've done a quick scan of issues Brian is nosy on to refresh my
> memory, and I'd say he's definitely been making positive contributions.
> I'm willing to volunteer to keep an eye on his triage work for a while
> if you grant him tracker privs.
>
>
Done for the username brian.curtin (email doesn't match the one Brian
emailed from so do let me know, Brian if this is the right username).
Welcome aboard!

-Brett


> Brian, I assume you'll be cognizant of Antoine's advice about making
> sure a bug really should be closed before closing it :)  Hanging out in
> #python-dev on freenode while working on issues can be helpful, as well,
> since you can quickly ask whoever is there for second opinions on
> particular bugs.
>
> --
> R. David Murray                                      www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/726f5ca0/attachment-0006.htm>

From brian.curtin at gmail.com  Thu Jan  7 02:59:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 19:59:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
	<bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
Message-ID: <cf9f31f21001061759m4fee1cc7g2d9fe8bbfd3cb38e@mail.gmail.com>

On Wed, Jan 6, 2010 at 19:28, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com>wrote:
>
>>
>> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
>> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
>> wrote:
>> > > On the topic of bugs that can be readily closed (literally), I've
>> recently
>> > > come across a number of issues which appear to be sitting in a patch
>> or
>> > > review stage, but their patches have been committed and the issue
>> remains
>> > > open. What is the best course of action there? I'd just go ahead and
>> close
>> > > the issue myself but I don't have tracker privileges.
>> > >
>> > >
>> > If a core developer is willing to step forward and vouch for you to get
>> > tracker privileges then I will give them to you. We are trying to give
>> out
>> > tracker privs w/ less time than required to get commit privileges. So as
>> > long as you have helped out on a few issues in a positive and correct
>> way
>> > that should be enough to get one of the regulars who perform triage to
>> > notice.
>> >
>> > -Brett
>>
>> I've done a quick scan of issues Brian is nosy on to refresh my
>> memory, and I'd say he's definitely been making positive contributions.
>> I'm willing to volunteer to keep an eye on his triage work for a while
>> if you grant him tracker privs.
>>
>>
> Done for the username brian.curtin (email doesn't match the one Brian
> emailed from so do let me know, Brian if this is the right username).
> Welcome aboard!
>
>
Yep, that's the one. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/62a95fa0/attachment-0006.htm>

From python at mrabarnett.plus.com  Thu Jan  7 04:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 07 Jan 2010 03:07:56 +0000
Subject: [Python-Dev] GIL required for _all_ Python calls?
Message-ID: <4B45500C.8090905@mrabarnett.plus.com>

Hi,

I've been wondering whether it's possible to release the GIL in the
regex engine during matching.

I know that it needs to have the GIL during memory-management calls, but
does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
an easy way to find out? Or is it just a case of checking the source
files for mentions of the GIL? The header file for PyList_New, for
example, doesn't mention it!

Thanks


From john.arbash.meinel at gmail.com  Thu Jan  7 04:25:48 2010
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 06 Jan 2010 21:25:48 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B45543C.2090200@gmail.com>

MRAB wrote:
> Hi,
> 
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out? Or is it just a case of checking the source
> files for mentions of the GIL? The header file for PyList_New, for
> example, doesn't mention it!
> 
> Thanks

Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
get concurrent updating of the value, and then the final value is wrong.
(two threads do 5+1 getting 6, rather than 7, and when the decref, you
end up at 4 rather than back at 5).

AFAIK, the only things that don't require the GIL are macro functions,
like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
example, will be increfing and setting the exception state, so certainly
needs the GIL to be held.

John
=:->



From benjamin at python.org  Thu Jan  7 04:32:17 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jan 2010 21:32:17 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45543C.2090200@gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com> <4B45543C.2090200@gmail.com>
Message-ID: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>

2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
> get concurrent updating of the value, and then the final value is wrong.
> (two threads do 5+1 getting 6, rather than 7, and when the decref, you
> end up at 4 rather than back at 5).

Correct.

>
> AFAIK, the only things that don't require the GIL are macro functions,
> like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> example, will be increfing and setting the exception state, so certainly
> needs the GIL to be held.

As a general rule, I would say, no Py* macros are safe without the gil
either (the exception being Py_END_ALLOW_THREADS), since they mutate
Python objects which must be protected.



-- 
Regards,
Benjamin


From guido at python.org  Thu Jan  7 05:29:03 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Jan 2010 20:29:03 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B45543C.2090200@gmail.com> 
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
Message-ID: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>

On Wed, Jan 6, 2010 at 7:32 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> > AFAIK, the only things that don't require the GIL are macro functions,
> > like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> > example, will be increfing and setting the exception state, so certainly
> > needs the GIL to be held.
>
> As a general rule, I would say, no Py* macros are safe without the gil
> either (the exception being Py_END_ALLOW_THREADS), since they mutate
> Python objects which must be protected.

That's keeping it on the safe side, since there are some macros like
PyString_AS_STRING() that are also safe, *if* you are owning at least
one reference to the string object.

At the same time, "no Py* macros" is not quite strong enough, since if
you called PyString_AS_STRING() before releasing the GIL but you don't
own a reference to the string object, the string might be deallocated
behind your back by another thread.

A better rule would be "you may access the memory buffer in a PyString
or PyUnicode object with the GIL released as long as you own a
reference to the string object." Everything else is out of bounds (or
not worth the bother).

--
--Guido van Rossum (python.org/~guido)


From yoann.padioleau at facebook.com  Thu Jan  7 08:24:42 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Wed, 6 Jan 2010 23:24:42 -0800
Subject: [Python-Dev] relation between Python.asdl and Tools/compiler/ast.txt
Message-ID: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>

Hi,

I would like to use astgen.py to generate python classes corresponding to the 
AST of something I have defined in a .asdl file, along the line of what is
apparently done for the python AST itself. I thought astgen.py would
take as an argument a .asdl file, but apparently it instead process a file
called ast.txt. Where does this file come from ? Is it generated from
Python.asdl ?



From johan.gill at agama.tv  Thu Jan  7 10:46:52 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 10:46:52 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
Message-ID: <4B45AD8C.5000405@agama.tv>

Hi devs,
the company where I work has done some work on Python, and the question 
is how this work, owned by the company, can be contributed to the 
community properly. Are there any license issues or other pitfalls we 
need to think about? I imagine that other companies have contributed 
before, so this is probably an already solved problem.

Regards
Johan Gill
Agama Technologies



From regebro at gmail.com  Thu Jan  7 12:12:17 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 12:12:17 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45AD8C.5000405@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
Message-ID: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
> Hi devs,
> the company where I work has done some work on Python, and the question is
> how this work, owned by the company, can be contributed to the community
> properly. Are there any license issues or other pitfalls we need to think
> about? I imagine that other companies have contributed before, so this is
> probably an already solved problem.

I'm not a license lawyer, but typically your company needs to give the
code to the community. Yes, it means it stops owning it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From hodgestar+pythondev at gmail.com  Thu Jan  7 12:28:01 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 7 Jan 2010 13:28:01 +0200
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <fb73205e1001070328j44ac747fu7232a89b559ad0d9@mail.gmail.com>

On Thu, Jan 7, 2010 at 1:12 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

This is incorrect.

The correct information is at http://www.python.org/psf/contrib/.

Schiavo
Simon


From swamy.sangamesh at gmail.com  Thu Jan  7 11:46:59 2010
From: swamy.sangamesh at gmail.com (swamy sangamesh)
Date: Thu, 7 Jan 2010 16:16:59 +0530
Subject: [Python-Dev] test_ctypes failure on AIX 5.3 using python-2.6.2 and
	libffi-3.0.9
Message-ID: <c3369a531001070246s441e159cg35b4cab4e3f65541@mail.gmail.com>

Hi All,

I built the python-2.6.2 with the latest libffi-3.0.9 in AIX 5.3 using xlc
compiler.
When i try to run the ctypes test cases, two failures are seen in
test_bitfields.

*test_ints (ctypes.test.test_bitfields.C_Test) ... FAIL
test_shorts (ctypes.test.test_bitfields.C_Test) ... FAIL*

I have attached the full test case result.

If i change the type c_int and c_short to c_unit and c_ushort of class
"BITS(Structure)" in file
test_bitfields.py then no failures are seen.

Has anyone faced the similar issue or any help is appreciated.


Thanks,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0006.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctype-testcases
Type: application/octet-stream
Size: 22996 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0006.obj>

From ncoghlan at gmail.com  Thu Jan  7 13:23:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 07 Jan 2010 22:23:11 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <4B45D22F.40404@gmail.com>

Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
>> Hi devs,
>> the company where I work has done some work on Python, and the question is
>> how this work, owned by the company, can be contributed to the community
>> properly. Are there any license issues or other pitfalls we need to think
>> about? I imagine that other companies have contributed before, so this is
>> probably an already solved problem.
> 
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

As Simon pointed out, while some organisations do work that way, the PSF
isn't one of them.

The PSF only requires that the code be contributed under a license that
then allows us to turn around and redistribute it under a different open
source license without requesting additional permission from the
copyright holder. For corporate contributions, I believe the contributor
agreement needs to be signed by an authorised agent of the company - the
place to check that would probably be psf at python.org (that's the email
address for the PSF board).

Assuming the subject line relates to the code that you would like to
contribute though, that particular change is unlikely to happen - 2.6 is
in maintenance mode and changing RLock from a Python implementation to
the faster C one is solidly in new feature territory. Although a
backport of the 3.2 C RLock implementation to 2.7 could be useful, I
doubt that backporting code provided by an existing committer would be
the subject of this query :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Thu Jan  7 14:11:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:11:27 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>

On Thu, Jan 7, 2010 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder.

Even if the contributed code as in this case is a method in an
existing file? How does that work, how do they keep ownership of one
method in the threading.py method? :-)

> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)

Ah. I probably misunderstood what the suggested contribution was.
Maybe it was a separate file, which I didn't get.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Thu Jan  7 14:15:09 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 07 Jan 2010 13:15:09 +0000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
Message-ID: <4B45DE5D.3030104@voidspace.org.uk>

On 07/01/2010 13:11, Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 13:23, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>    
>> As Simon pointed out, while some organisations do work that way, the PSF
>> isn't one of them.
>>
>> The PSF only requires that the code be contributed under a license that
>> then allows us to turn around and redistribute it under a different open
>> source license without requesting additional permission from the
>> copyright holder.
>>      
> Even if the contributed code as in this case is a method in an
> existing file? How does that work, how do they keep ownership of one
> method in the threading.py method? :-)
>
>    

When contributing code to Python all work remains copyright the original 
author. The combined work is copyright *everyone*. The PSF has a license 
to distribute it, which is all that is important.

How you meaningfully exercise your ownership over chunks of code is left 
for the reader to determine...

(i.e. copyright and ownership are legal terms that don't necessarily 
mean anything *practical* in these situations.)

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From stefan_ml at behnel.de  Thu Jan  7 14:30:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 14:30:16 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
Message-ID: <hi4nl7$2ej$1@ger.gmane.org>

Guido van Rossum, 07.01.2010 05:29:
> A better rule would be "you may access the memory buffer in a PyString
> or PyUnicode object with the GIL released as long as you own a
> reference to the string object." Everything else is out of bounds (or
> not worth the bother).

Is that a "yes" regarding the OP's original question about releasing the 
GIL during regexp searches?

Stefan



From regebro at gmail.com  Thu Jan  7 14:36:42 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:36:42 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DE5D.3030104@voidspace.org.uk>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
	<4B45DE5D.3030104@voidspace.org.uk>
Message-ID: <319e029f1001070536t49f76899j111212b02c14f192@mail.gmail.com>

On Thu, Jan 7, 2010 at 14:15, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> (i.e. copyright and ownership are legal terms that don't necessarily mean
> anything *practical* in these situations.)

OK, fair enough. :-)
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Thu Jan  7 15:05:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 15:05:23 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <hi4pn2$9so$1@ger.gmane.org>

MRAB, 07.01.2010 04:07:
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER

Py_UNICODE_TOLOWER looks safe to me at first glance.


> or PyErr_SetString?

Certainly not safe.


> Is there an easy way to find out?

Release it and fix any crashes? Note that this isn't a safe solution, 
though, as some GIL requiring code may be platform specific. So a better 
approach might be to extract any obviously problematic stuff from the 
existing code (such as any exception handling, explicit ref-counting or 
object creation), and *then* try to release the GIL.

Stefan



From solipsis at pitrou.net  Thu Jan  7 16:38:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 15:38:31 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <loom.20100107T163459-842@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out?

There is no "easy way" to do so. The only safe way is to examine all the
functions or macros you want to call with the GIL released, and assess whether
it is safe to call them. As already pointed out, no reference count should be
changed, and generally no mutable container should be accessed, except if that
container is known not to be referenced anywhere else (that would be the case
for e.g. a list that your function has created and is busy populating).

I agree that releasing the GIL when doing non-trivial regex searches is a
worthwhile research, so please don't give up immediately :-)

Regards

Antoine Pitrou.




From olemis at gmail.com  Thu Jan  7 19:24:59 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 7 Jan 2010 13:24:59 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
	<24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
Message-ID: <24ea26601001071024m2abd988fuc77f94845d57483a@mail.gmail.com>

On Mon, Jan 4, 2010 at 9:24 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
>> Thanks for the quick response.
>>
>> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> but maybe a
>> discussion could start about a new, less hacky, way of doing the same
>>
>
> I am strongly -1 for modifying the classes in ?traditional? unittest
> module [2]_ (except that I am strongly +1 for the package structure
> WITHOUT TOUCHING anything else ...) , and the more I think about it I
> am more convinced ... but anyway, this not a big deal (and in the end
> what I think is not that relevant either ... :o). So ...
>

IOW, if I had all the freedom to implement it, after the pkg structure
I'd do something like :

{{{
#!python

class TestResult:
    # Everything just the same
    def _is_relevant_tb_level(self, tb):
        return '__unittest' in tb.tb_frame.f_globals

class BetterTestResult(TestResult):
    # Further code ... maybe ;o)
    #
    def _is_relevant_tb_level(self, tb):
        # This or anything else you might want to do ;o)
        #
        globs = tb.tb_frame.f_globals
        is_relevant =  '__name__' in globs and \
            globs["__name__"].startswith("unittest")
        del globs
        return is_relevant
}}}

that's what inheritance is for ;o) ... but quite probably that's not
gonna happen, just a comment .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Ubuntu sustituye GIMP por F-Spot  -
http://feedproxy.google.com/~r/simelo-es/~3/-g48D6T6Ojs/ubuntu-sustituye-gimp-por-f-spot.html


From martin at v.loewis.de  Thu Jan  7 21:24:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:24:32 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi4nl7$2ej$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
	<hi4nl7$2ej$1@ger.gmane.org>
Message-ID: <4B464300.2020204@v.loewis.de>

>> A better rule would be "you may access the memory buffer in a PyString
>> or PyUnicode object with the GIL released as long as you own a
>> reference to the string object." Everything else is out of bounds (or
>> not worth the bother).
> 
> Is that a "yes" regarding the OP's original question about releasing the
> GIL during regexp searches?

No, because the regex engine may also operate on buffers that start
moving around when you release the GIL.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:27:09 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:27:09 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B46439D.9030604@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

I don't think that's possible. The regex engine can also operate on
objects whose representation may move in memory when you don't hold
the GIL (e.g. buffers that get mutated). Even if they stay in place -
if their contents changes, regex results may be confusing.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:31:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:31:21 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
Message-ID: <4B464499.4020305@v.loewis.de>

> I would like to use astgen.py to generate python classes corresponding to the 
> AST of something I have defined in a .asdl file, along the line of what is
> apparently done for the python AST itself. I thought astgen.py would
> take as an argument a .asdl file, but apparently it instead process a file
> called ast.txt. Where does this file come from ? Is it generated from
> Python.asdl ?

astgen.py is not used to process asdl files; ast.txt lives right next to
astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

HTH,
Martin


From foom at fuhm.net  Thu Jan  7 21:36:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 7 Jan 2010 15:36:37 -0500
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B46439D.9030604@v.loewis.de>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>

On Jan 7, 2010, at 3:27 PM, Martin v. L?wis wrote:

>> I've been wondering whether it's possible to release the GIL in the
>> regex engine during matching.
>
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated). Even if they stay in place -
> if their contents changes, regex results may be confusing.

It seems probably worthwhile to optimize for the common case of using  
the regexp engine on an immutable object of type "str" or "bytes", and  
allow releasing the GIL in *that* case, even if you have to keep it  
for the general case.

James

From martin at v.loewis.de  Thu Jan  7 21:45:42 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:45:42 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
Message-ID: <4B4647F6.2060309@v.loewis.de>

>>> I've been wondering whether it's possible to release the GIL in the
>>> regex engine during matching.
>>
>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated). Even if they stay in place -
>> if their contents changes, regex results may be confusing.
> 
> It seems probably worthwhile to optimize for the common case of using
> the regexp engine on an immutable object of type "str" or "bytes", and
> allow releasing the GIL in *that* case, even if you have to keep it for
> the general case.

Right. This problem was the one that I thought of first.

Thinking about these things is fairly difficult (to me, at least), so
I think I could only tell whether I would consider a patch thread-safe
that released the GIL around matching under selected circumstances -
if I had the patch available. I don't see any obvious reason (assuming
Guido's list of conditions holds - i.e. you are holding references to
everything you access).

Regards,
Martin


From ncoghlan at gmail.com  Thu Jan  7 21:48:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 08 Jan 2010 06:48:05 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DECB.7070907@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com> <4B45DECB.7070907@agama.tv>
Message-ID: <4B464885.40806@gmail.com>

Johan Gill wrote:
> Yes, it is the new RLock implementation.
> If I understood this correctly, we should make a patch against trunk if
> anything should be contributed.

Yep.

> Do you mean that we wouldn't need the paperwork for backporting the
> original patch committed to py3k?

Whether or not a contributor agreement was essential for this particular
contribution would depend on how much new code was needed for the
backport, but the bulk of the copyright on the C RLock code would remain
with Antoine regardless.

However, sorting through the legalities of the contributor agreement
really is the best way to make sure every is squared away nice and
neatly from a legal point of view.

After all, even if I was a lawyer (which I'm not, I'm just a developer
with an interest in licensing issues), I still wouldn't be *your* lawyer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Jan  7 21:43:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 20:43:17 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <loom.20100107T214211-130@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated).

Why is it a problem? If we get a buffer through the new buffer API, the object
should ensure that the representation isn't moved away until the buffer is 
released.

Regards

Antoine.




From martin at v.loewis.de  Thu Jan  7 21:59:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:59:29 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B464B31.7040406@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

Ok, here is another problem: SRE_OP_REPEAT uses PyObject_MALLOC,
which requires the GIL (it then also may call PyErr_NoMemory,
which also requires the GIL).

Regards,
Martin


From johan.gill at agama.tv  Thu Jan  7 14:16:59 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 14:16:59 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <4B45DECB.7070907@agama.tv>

On 01/07/2010 01:23 PM, Nick Coghlan wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder. For corporate contributions, I believe the contributor
> agreement needs to be signed by an authorised agent of the company - the
> place to check that would probably be psf at python.org (that's the email
> address for the PSF board).
>
> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)
>
> Regards,
> Nick.
>
>    
Yes, it is the new RLock implementation.
If I understood this correctly, we should make a patch against trunk if 
anything should be contributed.
Do you mean that we wouldn't need the paperwork for backporting the 
original patch committed to py3k?

Regards
Johan Gill
Agama Technologies



From yoann.padioleau at facebook.com  Thu Jan  7 22:07:55 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Thu, 7 Jan 2010 13:07:55 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464499.4020305@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
Message-ID: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>


On Jan 7, 2010, at 12:31 PM, Martin v. L?wis wrote:

>> I would like to use astgen.py to generate python classes corresponding to the 
>> AST of something I have defined in a .asdl file, along the line of what is
>> apparently done for the python AST itself. I thought astgen.py would
>> take as an argument a .asdl file, but apparently it instead process a file
>> called ast.txt. Where does this file come from ? Is it generated from
>> Python.asdl ?
> 
> astgen.py is not used to process asdl files; ast.txt lives right next to
> astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

Yes, I know that. That's why I asked about the relation between ast.txt and Python.adsl.
If internally the parser uses the .adsl, but expose as a reflection mechanism things
that were generated from ast.txt, then there could be a mismatch. Where does ast.txt comes from ? Shouldn't it be generated itself from Python.adsl ?

So we would have

Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py containing all the UnarySub, Expression, classes that represents a Python AST.



> 
> HTH,
> Martin



From martin at v.loewis.de  Thu Jan  7 22:11:36 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 07 Jan 2010 22:11:36 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <loom.20100107T214211-130@post.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
Message-ID: <4B464E08.5020703@v.loewis.de>

>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated).
> 
> Why is it a problem? If we get a buffer through the new buffer API, the object
> should ensure that the representation isn't moved away until the buffer is 
> released.

In 2.7, we currently get the buffer with bf_getreadbuffer. In 3.x, we have

    /* Release the buffer immediately --- possibly dangerous
       but doing something else would require some re-factoring
    */
    PyBuffer_Release(&view);


Even if we do use the new API, and correctly, it still might be
confusing if the contents of the buffer changes underneath.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 22:16:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 22:16:12 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
Message-ID: <4B464F1C.7020404@v.loewis.de>

>> astgen.py is not used to process asdl files; ast.txt lives right
>> next to astgen.py. Instead, the asdl file is processed by
>> Parser/asdl_c.py.
> 
> Yes, I know that. That's why I asked about the relation between
> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
> expose as a reflection mechanism things that were generated from
> ast.txt, then there could be a mismatch. Where does ast.txt comes
> from ? Shouldn't it be generated itself from Python.adsl ?

What you may not be aware of is that Tools/compiler (and the
compiler package that it builds on) are both unused and unmaintained.

If the package stops working correctly - tough luck.

> So we would have
> 
> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
> containing all the UnarySub, Expression, classes that represents a
> Python AST.

No - what actually happens in Python 3.x is this: both the compiler
package and Tools/compiler are removed.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 01:10:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 01:10:35 +0100
Subject: [Python-Dev] Improve open() to support reading file starting with
	an unicode BOM
Message-ID: <201001080110.35800.victor.stinner@haypocalc.com>

Hi,

Builtin open() function is unable to open an UTF-16/32 file starting with a 
BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
file starting with a BOM, read()/readline() returns also the BOM whereas the 
BOM should be "ignored".

See recent issues related to reading an UTF-8 text file including a BOM: #7185 
(csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with 
the UTF-8-SIG encoding, but it's possible to do better.

I propose to improve open() (TextIOWrapper) by using the BOM to choose the 
right encoding. I think that only files opened in read only mode should 
support this new feature. *Read* the BOM in a *write* only file would cause 
unexpected behaviours.

Since my proposition changes the result TextIOWrapper.read()/readline() for 
files starting with a BOM, we might introduce an option to open() to enable 
the new behaviour. But is it really needed to keep the backward compatibility?

I wrote a proof of concept attached to the issue #7651. My patch only changes 
the behaviour of TextIOWrapper for reading files starting with a BOM. It 
doesn't work yet if a seek() is used before the first read.

-- 
Victor Stinner
http://www.haypocalc.com/


From guido at python.org  Fri Jan  8 01:52:20 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 16:52:20 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>

I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
talk. And for the other two, perhaps it would make more sense to have
a separate encoding-guessing function that takes a binary stream and
returns a text stream wrapping it with the proper encoding?

--Guido

On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
> file starting with a BOM, read()/readline() returns also the BOM whereas the
> BOM should be "ignored".
>
> See recent issues related to reading an UTF-8 text file including a BOM: #7185
> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
> the UTF-8-SIG encoding, but it's possible to do better.
>
> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
> right encoding. I think that only files opened in read only mode should
> support this new feature. *Read* the BOM in a *write* only file would cause
> unexpected behaviours.
>
> Since my proposition changes the result TextIOWrapper.read()/readline() for
> files starting with a BOM, we might introduce an option to open() to enable
> the new behaviour. But is it really needed to keep the backward compatibility?
>
> I wrote a proof of concept attached to the issue #7651. My patch only changes
> the behaviour of TextIOWrapper for reading files starting with a BOM. It
> doesn't work yet if a seek() is used before the first read.
>
> --
> Victor Stinner
> http://www.haypocalc.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Fri Jan  8 03:23:08 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 02:23:08 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <4B46970C.9010306@mrabarnett.plus.com>

Guido van Rossum wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
> 
Alternatively, have a universal UTF-8/16/32 encoding, ie one that 
expects UTF-8,
with or without BOM, or UTF-16/32 with BOM.
> 
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
>> See recent issues related to reading an UTF-8 text file including a BOM: #7185
>> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
>> the UTF-8-SIG encoding, but it's possible to do better.
>>
>> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
>> right encoding. I think that only files opened in read only mode should
>> support this new feature. *Read* the BOM in a *write* only file would cause
>> unexpected behaviours.
>>
>> Since my proposition changes the result TextIOWrapper.read()/readline() for
>> files starting with a BOM, we might introduce an option to open() to enable
>> the new behaviour. But is it really needed to keep the backward compatibility?
>>
>> I wrote a proof of concept attached to the issue #7651. My patch only changes
>> the behaviour of TextIOWrapper for reading files starting with a BOM. It
>> doesn't work yet if a seek() is used before the first read.
>>


From nick.bastin at gmail.com  Fri Jan  8 04:11:03 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Thu, 7 Jan 2010 22:11:03 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
Message-ID: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>

I think this problem probably needs to move over to distutils-sig, as
it doesn't seem to be specific to the way that Python itself uses
distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
linux and solaris and automatically adds '.' to the library_dirs, and
I suspect it just needs to do this on FreeBSD as well (adding bsd to
the list of platforms for which this is performed "solves" the
problem, but I don't pretend to know enough about either distutils or
freebsd to determine if this is the correct solution).

--
Nick


From glyph at twistedmatrix.com  Fri Jan  8 04:34:36 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 7 Jan 2010 22:34:36 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>



On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>> 
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".

> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

It *is* crazy, but unfortunately rather common.  Wikipedia has a good description of the issues: <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as being UTF-8, so it's become a convention to do that.  That's not good enough, so you need to guess the encoding as well to make sure, but if there is a BOM and you can otherwise verify that the file is probably UTF-8 encoded, you should discard it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/1bc40870/attachment-0006.htm>

From tseaver at palladion.com  Fri Jan  8 05:17:28 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 Jan 2010 23:17:28 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <hi6bko$d0h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
> linux and solaris and automatically adds '.' to the library_dirs, and
> I suspect it just needs to do this on FreeBSD as well (adding bsd to
> the list of platforms for which this is performed "solves" the
> problem, but I don't pretend to know enough about either distutils or
> freebsd to determine if this is the correct solution).

I wouldn't say it needed discussion on the SIG:  just create a bug
report, with the tentative patch you have worked out, and get it
assigned to Tarek.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGsdQACgkQ+gerLs4ltQ5BMQCgtV8snMXH/6dDwgdN4sIJljLd
koYAoKq6c0tKsRSrITHcygu4Od9FVzF5
=BJaE
-----END PGP SIGNATURE-----



From guido at python.org  Fri Jan  8 05:21:04 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 20:21:04 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
Message-ID: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>

On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>
>
> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
>
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>
> file starting with a BOM, read()/readline() returns also the BOM whereas the
>
> BOM should be "ignored".
>
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>
> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
> description of the issues:
> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
> being UTF-8, so it's become a convention to do that. ?That's not good
> enough, so you need to guess the encoding as well to make sure, but if there
> is a BOM and you can otherwise verify that the file is probably UTF-8
> encoded, you should discard it.

That doesn't make sense. If the file isn't UTF-8 you can't see the
BOM, because the BOM itself is UTF-8-encoded.

(And yes, I know this happens. Doesn't mean we need to auto-guess by
default; there are lots of issues e.g. what should happen after
seeking to offset 0?)

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Fri Jan  8 07:06:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 08 Jan 2010 15:06:16 +0900
Subject: [Python-Dev] Improve open() to support reading file
	starting	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
 > talk.

That doesn't stop many applications from doing it.  Python should
perhaps<wink,nudge> not produce UTF-8 + BOM without a disclaimer of
indemnification against all resulting damage, signed in blood, from
the user for each instance.

But it should do something sane when reading such files.  I can't
really see any harm in throwing it away, especially since use of
ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
IIRC.






From tseaver at palladion.com  Fri Jan  8 07:12:12 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 01:12:12 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <hi6ibr$qag$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>
>> BOM should be "ignored".
>>
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.
> 
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

The BOM should not be seekeable if the file is opened with the proposed
"guess encoding from BOM" mode:  it isn't properly part of the stream at
all in that case.

A UTF-8 BOM is an absurditiy, but it exists *everywhere* in the wild:
Python would do wll to make it as easy as possible to consume such
files, as well as the non-insane versions (UTF-16 / UTF-32 BOMs).  In
the best of all possible worlds, I would just try opening the file so:

  f = open('/path/to/file', 'r', encoding="DWIFM")

and any BOM present would set the encoding for the remainder of the stream..



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGzLsACgkQ+gerLs4ltQ5+cwCdGfycPdj6+cPfD23vH644SpHL
sI0AoLGD7nfgMEJdJhBr90yjQQHfDgcJ
=js+2
-----END PGP SIGNATURE-----



From glyph at twistedmatrix.com  Fri Jan  8 08:55:27 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 8 Jan 2010 02:55:27 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>


On Jan 7, 2010, at 11:21 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> 
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>> 
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>> 
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.  If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal.  It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I think it's pretty clear that the BOM should still be skipped in that case ...



From martin at v.loewis.de  Fri Jan  8 10:05:17 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:05:17 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <4B46F54D.9090103@v.loewis.de>

>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I think what Glyph meant is this: if a file starts with the UTF-8
signature, assume it's UTF-8. Then validate the assumption against the
rest of the file also, and then process it as UTF-8. If the rest clearly
is not UTF-8, assume that the UTF-8 signature is bogus.

I understood this proposal as a general processing guideline, not
something the io library should do (but, say, a text editor).

FWIW, I'm personally in favor of using the UTF-8 signature. If people
consider them crazy talk, that may be because UTF-8 can't possibly have
a byte order - hence I call it a signature, not the BOM. As a signature,
I don't consider it crazy at all. There is a long tradition of having
magic bytes in files (executable files, Postscript, PDF, ... - see
/etc/magic). Having a magic byte sequence for plain text to denote the
encoding is useful and helps reducing moji-bake. This is the reason it's
used on Windows: notepad would normally assume that text is in the ANSI
code page, and for compatibility, it can't stop doing that. So the UTF-8
signature gives them an exit strategy.

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:06:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:06:34 +0100
Subject: [Python-Dev] Improve open() to support reading file	starting
 with an unicode BOM
In-Reply-To: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B46F59A.5060905@v.loewis.de>

> But it should do something sane when reading such files.  I can't
> really see any harm in throwing it away, especially since use of
> ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
> IIRC.

And indeed it does, when you open the file in the utf-8-sig encoding.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 10:08:30 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 10:08:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46970C.9010306@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<4B46970C.9010306@mrabarnett.plus.com>
Message-ID: <201001081008.30904.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 03:23:08, MRAB a ?crit :
> Guido van Rossum wrote:
> > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> > talk. And for the other two, perhaps it would make more sense to have
> > a separate encoding-guessing function that takes a binary stream and
> > returns a text stream wrapping it with the proper encoding?
> 
> Alternatively, have a universal UTF-8/16/32 encoding, ie one that
> expects UTF-8,
> with or without BOM, or UTF-16/32 with BOM.

Do you mean open(filename, encoding="BOM")? I suppose that "BOM" would be a 
magical value specific to read a text file (open(filename, "r")), not a real 
codec?

Otherwise which encoding should be used for open(filename, "w", 
encoding="BOM")?

-- 
Victor Stinner
http://www.haypocalc.com/


From martin at v.loewis.de  Fri Jan  8 10:10:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:10:23 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with	an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <4B46F67F.60604@v.loewis.de>

> Builtin open() function is unable to open an UTF-16/32 file starting with a 
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
> file starting with a BOM, read()/readline() returns also the BOM whereas the 
> BOM should be "ignored".

It depends. If you use the utf-8-sig encoding, it *will* ignore the
UTF-8 signature.

> Since my proposition changes the result TextIOWrapper.read()/readline() for 
> files starting with a BOM, we might introduce an option to open() to enable 
> the new behaviour. But is it really needed to keep the backward compatibility?

Absolutely. And there is no need to produce a new option, but instead
use the existing options: define an encoding that auto-detects the
encoding from the family of BOMs. Maybe you call it encoding="sniff".

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:11:51 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:11:51 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <4B46F6D7.1050301@v.loewis.de>

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.

I'm fairly skeptical that anybody on distutils SIG is interested in
details of the Python build process...

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 11:27:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:27:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <201001081127.44044.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
(...)
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I wrote a new version of my patch (version 3):

 * don't change the default behaviour: use open(filename, encoding="BOM") to 
check the BOM is there is any
 * fix for seek(0): always ignore the BOM
 * add an unit test: check that the right encoding is detect, but also the the 
BOM is ignored (especially after a seek(0))

BOM encoding doesn't work for writing into a file, so open(filename, "w", 
encoding="BOM") raises a ValueError.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:31:37 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:31:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <201001081131.37492.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 01:52:20, Guido van Rossum a ?crit :
> And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

I choosed to modify open()+TextIOWrapper instead of writing a new function 
because I would like to avoid an extra read operation (syscall) on the file. 
With my implementation, no specific read operation is needed to detect the 
BOM. The BOM is simply checked in the first _read_chunk().

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:40:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:40:28 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Improve_open=28=29_to_support_reading?=
 =?iso-8859-1?q?_file_starting_with=09an_unicode_BOM?=
In-Reply-To: <4B46F67F.60604@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
Message-ID: <201001081140.28124.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
> > Builtin open() function is unable to open an UTF-16/32 file starting with
> > a BOM if the encoding is not specified (raise an unicode error). For an
> > UTF-8 file starting with a BOM, read()/readline() returns also the BOM
> > whereas the BOM should be "ignored".
> 
> It depends. If you use the utf-8-sig encoding, it *will* ignore the
> UTF-8 signature.

Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
remove the BOM after the first read (much harder if you use a module like 
ConfigParser or csv).

> > Since my proposition changes the result TextIOWrapper.read()/readline()
> > for files starting with a BOM, we might introduce an option to open() to
> > enable the new behaviour. But is it really needed to keep the backward
> > compatibility?
> 
> Absolutely. And there is no need to produce a new option, but instead
> use the existing options: define an encoding that auto-detects the
> encoding from the family of BOMs. Maybe you call it encoding="sniff".

Good idea, I choosed open(filename, encoding="BOM").

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Fri Jan  8 15:27:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:27:42 +0000 (UTC)
Subject: [Python-Dev] GIL required for _all_ Python calls?
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de>
Message-ID: <hi7fcu$gs7$1@ger.gmane.org>

Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
> 
> Even if we do use the new API, and correctly, it still might be
> confusing if the contents of the buffer changes underneath.

Well, no more confusing than when you compute a SHA1 hash or zlib-
compress the buffer, is it?

Regards

Antoine




From solipsis at pitrou.net  Fri Jan  8 15:34:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:34:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <loom.20100108T153305-228@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any

Well, I think if we implement this the default behaviour *should* be changed.
It looks a bit senseless to have two different "auto-choose" options, one with
encoding=None and one with encoding="BOM".

Regards

Antoine.




From guido at python.org  Fri Jan  8 16:48:44 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:48:44 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi7fcu$gs7$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de> 
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de> <hi7fcu$gs7$1@ger.gmane.org>
Message-ID: <ca471dc21001080748lbc18bbx4c4ec2d3f4f027e@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
>>
>> Even if we do use the new API, and correctly, it still might be
>> confusing if the contents of the buffer changes underneath.
>
> Well, no more confusing than when you compute a SHA1 hash or zlib-
> compress the buffer, is it?

That depends. Algorithms that make exactly one pass over the buffer
will run fine (maybe producing a meaningless result). But the regex
matcher may scan the buffer repeatedly (for backtracking purposes) and
it would take a considerable analysis to prove that cannot mess up its
internal data structures if the data underneath changes. (I give it a
decent chance that it's fine, but since it was written without ever
considering this possibility I'm not 100% sure.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:52:48 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:52:48 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
Message-ID: <ca471dc21001080752r312dd47fx32486a95336160ec@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:55 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
> I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.

And I'm saying that it is, with as much certainty as we can ever guess
the encoding of a file.

> If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal. ?It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

Sure, a Latin-1-encoded file could start with the same pattern that is
a UTF-8-encoded BOM. But at that point, a UTF-16-encoded file is also
valid Latin-1.

The question was in the context of encoding-guessing; if we're
guessing, a UTF-8-encoded BOM cannot signify anything else but UTF-8.
(Ditto for UTF-16 and UTF-32 BOMs.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:54:15 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:54:15 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100108T153305-228@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
Message-ID: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>
>> I wrote a new version of my patch (version 3):
>>
>> ?* don't change the default behaviour: use open(filename, encoding="BOM") to
>> check the BOM is there is any
>
> Well, I think if we implement this the default behaviour *should* be changed.
> It looks a bit senseless to have two different "auto-choose" options, one with
> encoding=None and one with encoding="BOM".

Well there *are* two different auto options: use the environment
variables (LANG etc.) or inspect the contents of the file. I think it
would be useful to have ways to specify both.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:56:46 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:56:46 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <ca471dc21001080756p5cb9f560x2a4443efa441fa7b@mail.gmail.com>

On Fri, Jan 8, 2010 at 1:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that. ?That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>>
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
>
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.
>
> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
>
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Sure. I said "crazy talk" only to stir up discussion. Which worked. :-)

Also, I don't want Python's default behavior to change -- sniffing the
encoding should be a separate option.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:59:45 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:59:45 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi6ibr$qag$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<hi6ibr$qag$1@ger.gmane.org>
Message-ID: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
> The BOM should not be seekeable if the file is opened with the proposed
> "guess encoding from BOM" mode: ?it isn't properly part of the stream at
> all in that case.

This feels about right to me. There are still questions though:
immediately after opening a file with a BOM, what should .tell()
return? And regardless of that, .seek(0) should put the file in that
same initial state.

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Fri Jan  8 17:03:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:03:07 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <loom.20100108T170053-283@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.




From mal at egenix.com  Fri Jan  8 17:25:22 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 17:25:22 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <4B475C72.1010207@egenix.com>

Guido van Rossum wrote:
> On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>>
>>> I wrote a new version of my patch (version 3):
>>>
>>>  * don't change the default behaviour: use open(filename, encoding="BOM") to
>>> check the BOM is there is any
>>
>> Well, I think if we implement this the default behaviour *should* be changed.
>> It looks a bit senseless to have two different "auto-choose" options, one with
>> encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Shouldn't this encoding guessing be a separate function that you call
on either a file or a seekable stream ?

After all, detecting encodings is just as useful to have for non-file
streams. You'd then avoid having to stuff everything into
a single function call and also open up the door for more complex
application specific guess work or defaults.

The whole process would then have two steps:

 1. guess encoding

  import codecs
  encoding = codecs.guess_file_encoding(filename)

 2. open the file with the found encoding

  f = open(filename, encoding=encoding)

For seekable streams f, you'd have:

 1. guess encoding

  import codecs
  encoding = codecs.guess_stream_encoding(f)

 2. wrap the stream with a reader for the found encoding

  reader_class = codecs.getreader(encoding)
  g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Jan  8 17:31:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:31:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <loom.20100108T171644-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver <at> palladion.com>
wrote:
> > The BOM should not be seekeable if the file is opened with the proposed
> > "guess encoding from BOM" mode:  it isn't properly part of the stream at
> > all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return?

tell() in the context of text I/O is specified to return an "opaque cookie". So
whatever value it returns would probably be fine, as long as seeking to that
value leaves the file in an acceptable state.

Rewinding (seeking to 0) in the presence of a BOM is already reasonably
supported by the TextIOWrapper object:

>>> dec = codecs.getincrementaldecoder('utf-16')()
>>> dec.decode(b'\xff\xfea\x00b\x00')
'ab'
>>> dec.decode(b'\xff\xfea\x00b\x00')
'\ufeffab'
>>> 
>>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
>>> f = io.TextIOWrapper(bio, encoding='utf-16')
>>> f.read()
'ab'
>>> f.seek(0)
0
>>> f.read()
'ab'

There are tests for this in test_io.py (test_encoded_writes, line 1929, and
test_append_bom and test_seek_bom, line 2045).

Regards

Antoine.




From python at mrabarnett.plus.com  Fri Jan  8 17:47:18 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 16:47:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081127.44044.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <4B476196.4080409@mrabarnett.plus.com>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
> (...)
>> (And yes, I know this happens. Doesn't mean we need to auto-guess by
>> default; there are lots of issues e.g. what should happen after
>> seeking to offset 0?)
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any
>  * fix for seek(0): always ignore the BOM
>  * add an unit test: check that the right encoding is detect, but also the the 
> BOM is ignored (especially after a seek(0))
> 
> BOM encoding doesn't work for writing into a file, so open(filename, "w", 
> encoding="BOM") raises a ValueError.
> 
I think it's similar to universal newline mode. You can tell it that
you're reading UTF-something-encoded text (common forms only).

The preference is UTF-8, but it could be UTF-8-sig (from Windows), or
possibly UTF-16/32, which really need a BOM because there are multiple
bytes per codepoint, so the bytes could be big-endian or little-endian.

The BOM (or signature) tells you what the encoding is, defaulting to
UTF-8 if there's none. If it subsequently raises a DecodeError, then
so be it!

Maybe there should also be a way of determining what encoding it decided
it was, so that you can then write a new file in that same encoding.


From status at bugs.python.org  Fri Jan  8 18:07:13 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jan 2010 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100108170714.014B078669@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/01/10 - 01/08/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2544 open (+27) / 16937 closed (+15) / 19481 total (+42)

Open issues with patches:  1017

Average duration of open issues: 708 days.
Median duration of open issues: 464 days.

Open Issues Breakdown
   open  2509 (+27)
pending    34 ( +0)

Issues Created Or Reopened (43)
_______________________________

Extended slicing with classic class behaves strangely            01/07/10
       http://bugs.python.org/issue7532    reopened mark.dickinson                
       patch                                                                   

optparse library documentation has an insignificant formatting i 01/01/10
CLOSED http://bugs.python.org/issue7618    created  vazovsky                      
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6           01/01/10
CLOSED http://bugs.python.org/issue7619    created  djc                           
                                                                               

Vim syntax highlight                                             01/02/10
       http://bugs.python.org/issue7620    created  july                          
       patch                                                                   

Test issue                                                       01/02/10
CLOSED http://bugs.python.org/issue7621    created  georg.brandl                  
                                                                               

[patch] improve unicode methods: split() rsplit() and replace()  01/03/10
       http://bugs.python.org/issue7622    created  flox                          
       patch                                                                   

PropertyType missing in Lib/types.py                             01/03/10
CLOSED http://bugs.python.org/issue7623    created  wplappert                     
                                                                               

isinstance(... ,collections.Callable) fails with oldstyle class  01/03/10
       http://bugs.python.org/issue7624    created  rgammans                      
                                                                               

bytearray needs more tests for "b.some_method()[0] is not b"     01/03/10
       http://bugs.python.org/issue7625    created  flox                          
       patch                                                                   

Entity references without semicolon in HTMLParser                01/03/10
CLOSED http://bugs.python.org/issue7626    created  stefan.schweizer              
                                                                               

mailbox.MH.remove() lock handling is broken                      01/04/10
       http://bugs.python.org/issue7627    created  sraustein                     
                                                                               

round() doesn't work correctly in 3.1.1                          01/04/10
CLOSED http://bugs.python.org/issue7628    created  bkovt                         
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs" 01/04/10
CLOSED http://bugs.python.org/issue7629    created  popelkopp                     
                                                                               

Strange behaviour of decimal.Decimal                             01/04/10
CLOSED http://bugs.python.org/issue7630    created  parmax                        
                                                                               

undefined label: bltin-file-objects                              01/04/10
CLOSED http://bugs.python.org/issue7631    created  ezio.melotti                  
                                                                               

dtoa.c: oversize b in quorem                                     01/04/10
       http://bugs.python.org/issue7632    created  skrah                         
                                                                               

decimal.py: type conversion in context methods                   01/04/10
       http://bugs.python.org/issue7633    created  skrah                         
       patch, easy                                                             

next/previous links in documentation skip some sections          01/05/10
CLOSED http://bugs.python.org/issue7634    created  gagenellina                   
                                                                               

19.6 xml.dom.pulldom doc: stub?                                  01/05/10
       http://bugs.python.org/issue7635    created  tjreedy                       
                                                                               

Add a set update action to optparse                              01/05/10
CLOSED http://bugs.python.org/issue7636    created  hardkrash                     
       patch                                                                   

Improve 19.5. xml.dom.minidom doc                                01/05/10
       http://bugs.python.org/issue7637    created  tjreedy                       
                                                                               

Counterintuitive str.splitlines() inconsistency.                 01/05/10
CLOSED http://bugs.python.org/issue7638    created  vencabot_teppoo               
                                                                               

bdist_msi fails on files with long names                         01/05/10
       http://bugs.python.org/issue7639    created  mmm77                         
                                                                               

buffered io seek() buggy                                         01/05/10
       http://bugs.python.org/issue7640    created  pakal                         
       patch                                                                   

Built-in Formatter accepts undocumented presentation type        01/06/10
CLOSED http://bugs.python.org/issue7641    created  lrekucki                      
                                                                               

[patch] Minor improvement in os.system doc                       01/06/10
       http://bugs.python.org/issue7642    created  Val                           
       patch                                                                   

What is an ASCII linebreak?                                      01/06/10
       http://bugs.python.org/issue7643    created  flox                          
                                                                               

bug in nntplib.body() method with possible fix                   01/06/10
       http://bugs.python.org/issue7644    created  mdmullins                     
       easy                                                                    

test_distutils fails on Windows XP                               01/06/10
       http://bugs.python.org/issue7645    created  austin987                     
                                                                               

test_telnetlib fails in Windows XP                               01/06/10
       http://bugs.python.org/issue7646    created  austin987                     
                                                                               

Add statvfs flags to the posix module                            01/06/10
       http://bugs.python.org/issue7647    created  ajax at redhat.com               
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                 01/06/10
       http://bugs.python.org/issue7648    created  austin987                     
       easy                                                                    

"u'%c' % char" broken for chars in range '\x80'-'\xFF'           01/07/10
       http://bugs.python.org/issue7649    created  ezio.melotti                  
       patch, easy                                                             

test_uuid is invalid                                             01/07/10
       http://bugs.python.org/issue7650    created  austin987                     
                                                                               

Python3: guess text file charset using the BOM                   01/07/10
       http://bugs.python.org/issue7651    created  haypo                         
       patch                                                                   

Merge C version of decimal into py3k.                            01/07/10
       http://bugs.python.org/issue7652    created  mark.dickinson                
                                                                               

Fix description of the way the PythonPath Windows registry key w 01/07/10
CLOSED http://bugs.python.org/issue7653    created  BoarGules                     
       patch, needs review                                                     

[patch] Enable additional bytes and memoryview tests.            01/07/10
       http://bugs.python.org/issue7654    created  flox                          
       patch                                                                   

PEP 374 Title usage & script redirection typo fixes              01/07/10
CLOSED http://bugs.python.org/issue7655    created  bab9e9                        
       patch                                                                   

test_hashlib fails on some installations (specifically Neal's re 01/08/10
       http://bugs.python.org/issue7656    created  r.david.murray                
                                                                               

test_ctypes failure on AIX 5.3                                   01/08/10
       http://bugs.python.org/issue7657    created  mallayya                      
       patch                                                                   

OS X pythonw.c compile error with 10.4 or earlier deployment tar 01/08/10
       http://bugs.python.org/issue7658    created  ned.deily                     
                                                                               

Problems with attribute assignment on object instances           01/08/10
       http://bugs.python.org/issue7659    created  pakal                         
                                                                               



Issues Now Closed (40)
______________________

shutil fails when copying to NTFS in Linux                        762 days
       http://bugs.python.org/issue1545    benjamin.peterson             
       patch                                                                   

Test                                                              751 days
       http://bugs.python.org/issue1619    georg.brandl                  
                                                                               

Windows Registry issue with 2.5.2 AMD64 msi                       645 days
       http://bugs.python.org/issue2539    loewis                        
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     618 days
       http://bugs.python.org/issue2698    Daniel26                      
                                                                               

optparse print_usage(),.. methods are not documented              542 days
       http://bugs.python.org/issue3340    ezio.melotti                  
                                                                               

reference leaks in test_distutils                                 500 days
       http://bugs.python.org/issue3660    ezio.melotti                  
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                          20 days
       http://bugs.python.org/issue3745    lemburg                       
       26backport                                                              

Add Option to Bind to a Local IP Address in httplib.py            464 days
       http://bugs.python.org/issue3972    gregory.p.smith               
       patch, easy                                                             

no reference for optparse methods                                 388 days
       http://bugs.python.org/issue4635    ezio.melotti                  
                                                                               

PyArg_Parse* should raise TypeError for float parsed with intege  339 days
       http://bugs.python.org/issue5080    mark.dickinson                
       patch                                                                   

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()              282 days
       http://bugs.python.org/issue5576    mark.dickinson                
       patch                                                                   

PyFrame_GetLineNumber                                             244 days
       http://bugs.python.org/issue5954    georg.brandl                  
       patch                                                                   

PyCode_NewEmpty                                                   244 days
       http://bugs.python.org/issue5959    georg.brandl                  
       patch                                                                   

Add non-command help topics to help completion of cmd.Cmd         241 days
       http://bugs.python.org/issue5991    georg.brandl                  
       patch                                                                   

make error                                                        231 days
       http://bugs.python.org/issue6067    georg.brandl                  
                                                                               

no longer possible to hash arrays                                 229 days
       http://bugs.python.org/issue6071    benjamin.peterson             
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           173 days
       http://bugs.python.org/issue6511    r.david.murray                
                                                                               

use different mechanism for pythonw on osx                        127 days
       http://bugs.python.org/issue6834    ned.deily                     
       easy                                                                    

Py3k doc: "from __future__ import division" not necessary          32 days
       http://bugs.python.org/issue7432    ezio.melotti                  
                                                                               

cPickle: stack underflow in load_pop()                             31 days
       http://bugs.python.org/issue7455    pitrou                        
       patch                                                                   

crash in str.rfind() with an invalid start value                   25 days
       http://bugs.python.org/issue7458    pitrou                        
       patch                                                                   

Implement fastsearch algorithm for rfind/rindex                    26 days
       http://bugs.python.org/issue7462    ezio.melotti                  
       patch                                                                   

GZipFile.readline too slow                                         20 days
       http://bugs.python.org/issue7471    pitrou                        
       patch                                                                   

ssl module documentation: SSLSocket.unwrap description shown twi    5 days
       http://bugs.python.org/issue7592    georg.brandl                  
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            7 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

optparse library documentation has an insignificant formatting i    2 days
       http://bugs.python.org/issue7618    ezio.melotti                  
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6              1 days
       http://bugs.python.org/issue7619    djc                           
                                                                               

Test issue                                                          0 days
       http://bugs.python.org/issue7621    ezio.melotti                  
                                                                               

PropertyType missing in Lib/types.py                                0 days
       http://bugs.python.org/issue7623    benjamin.peterson             
                                                                               

Entity references without semicolon in HTMLParser                   2 days
       http://bugs.python.org/issue7626    r.david.murray                
                                                                               

round() doesn't work correctly in 3.1.1                             0 days
       http://bugs.python.org/issue7628    benjamin.peterson             
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs"    0 days
       http://bugs.python.org/issue7629    r.david.murray                
                                                                               

Strange behaviour of decimal.Decimal                                0 days
       http://bugs.python.org/issue7630    mark.dickinson                
                                                                               

undefined label: bltin-file-objects                                 0 days
       http://bugs.python.org/issue7631    pitrou                        
                                                                               

next/previous links in documentation skip some sections             0 days
       http://bugs.python.org/issue7634    georg.brandl                  
                                                                               

Add a set update action to optparse                                 3 days
       http://bugs.python.org/issue7636    r.david.murray                
       patch                                                                   

Counterintuitive str.splitlines() inconsistency.                    0 days
       http://bugs.python.org/issue7638    flox                          
                                                                               

Built-in Formatter accepts undocumented presentation type           0 days
       http://bugs.python.org/issue7641    eric.smith                    
                                                                               

Fix description of the way the PythonPath Windows registry key w    0 days
       http://bugs.python.org/issue7653    r.david.murray                
       patch, needs review                                                     

PEP 374 Title usage & script redirection typo fixes                 0 days
       http://bugs.python.org/issue7655    georg.brandl                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 22 [patch] improve unicode methods: split() rsplit() and replace()    5 days
open    http://bugs.python.org/issue7622   

 14 Test suite emits many DeprecationWarnings when -3 is enabled      91 days
open    http://bugs.python.org/issue7092   

 13 unicode_escape codec does not escape quotes                        8 days
open    http://bugs.python.org/issue7615   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    0 days
open    http://bugs.python.org/issue7658   

  8 Merge C version of decimal into py3k.                              1 days
open    http://bugs.python.org/issue7652   

  8 "u'%c' % char" broken for chars in range '\x80'-'\xFF'             2 days
open    http://bugs.python.org/issue7649   

  8 decimal.py: type conversion in context methods                     4 days
open    http://bugs.python.org/issue7633   

  8 Patch for [ 735515 ] urllib2 should cache 301 redir              906 days
open    http://bugs.python.org/issue1755841

  7 Test issue                                                         0 days
closed  http://bugs.python.org/issue7621   

  7 Cannot use both read and readline method in same ZipExtFile obj    8 days
open    http://bugs.python.org/issue7610   




From tseaver at palladion.com  Fri Jan  8 22:09:54 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:09:54 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <hi86v3$3j5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> The BOM should not be seekeable if the file is opened with the proposed
>> "guess encoding from BOM" mode:  it isn't properly part of the stream at
>> all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return? And regardless of that, .seek(0) should put the file in that
> same initial state.

I think the behavior should be something like:

 >>> f = open('/path/to/maybe-BOM-encoded-file', 'r', encoding='BOM')
 >>> f.tell()
 0L
 >>> f.seek(-1)
 >>> f.tell() # count of unicode chars in decoded stream
 45L
 >>> f.seek(0)
 >>> f.read(1) # read first unicode char decoded from stream.
 'A'

In other words, the BOM is not readable / seekable at all:  it is
invisible to the consumer of the decoded stream.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHnyIACgkQ+gerLs4ltQ6s3QCgznD+7FbUzfCbe5TS6OcoXjMg
rdgAoJAMEXe2xwLCIwJaZ6XA6rVyTIAi
=oXb3
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:19:10 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:19:10 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B475C72.1010207@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com>
Message-ID: <hi87gg$8ge$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:

> Shouldn't this encoding guessing be a separate function that you call
> on either a file or a seekable stream ?
> 
> After all, detecting encodings is just as useful to have for non-file
> streams.

Other stream sources typically have out-of-band ways to signal the
encoding:  only when reading from the filesystem do we pretty much
*have* to guess, and in that case the BOM / signature is the best
heuristic we have.  Also, some non-file streams are not seekable, and so
can't be guessed via a pre-pass.

> You'd then avoid having to stuff everything into
> a single function call and also open up the door for more complex
> application specific guess work or defaults.
> 
> The whole process would then have two steps:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_file_encoding(filename)

Filename is not enough information:  or do you mean that API to actually
open the stream?

>  2. open the file with the found encoding
> 
>   f = open(filename, encoding=encoding)
> 
> For seekable streams f, you'd have:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_stream_encoding(f)
> 
>  2. wrap the stream with a reader for the found encoding
> 
>   reader_class = codecs.getreader(encoding)
>   g = reader_class(f)
> 


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoU4ACgkQ+gerLs4ltQ5o3QCeLOJ7J91E+5f66vhgu1BUhYh4
9UgAnR2IeCd0BCsPez8ZilGNHJfhRn3Y
=SoPb
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:14:59 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:14:59 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <hi878l$7os$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:

>>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that.  That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
> 
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.

If the programmer opens the file using a "guess using the BOM" encoding,
 Python should *not* attempt to verify that the file is properly
encoded:  it should check for (and consume) any BOM, and then return a
stream which uses the encoding inferred from the BOM.  Any errors should
be handled later, when characters are read, exactly as if the file had
been opened with the same encoding guessed from the BOM.

> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
> 
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Agreed.  Having that marker at the start of the file makes interop with
other tools *much* easier.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoFMACgkQ+gerLs4ltQ73dACffwUfyh6Q9vUnKYf367QFjNcU
RRMAoNuKCWEx7j+MSdTv+UjhAPynBc14
=uAX6
-----END PGP SIGNATURE-----



From eric at trueblade.com  Fri Jan  8 22:40:47 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 8 Jan 2010 16:40:47 -0500 (EST)
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
Message-ID: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>

>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
>>
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
>
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

But what if the file were in (for example) a zip file? I think you
definitely want to have access to this functionality outside of open().

Eric.



From foom at fuhm.net  Fri Jan  8 22:49:23 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 8 Jan 2010 16:49:23 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de> <hi878l$7os$1@ger.gmane.org>
Message-ID: <D481E750-3EE7-4498-9959-B4E34E02FFC6@fuhm.net>

On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote:
>> I understood this proposal as a general processing guideline, not
>> something the io library should do (but, say, a text editor).
>>
>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly  
>> have
>> a byte order - hence I call it a signature, not the BOM. As a  
>> signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote  
>> the
>> encoding is useful and helps reducing moji-bake. This is the reason  
>> it's
>> used on Windows: notepad would normally assume that text is in the  
>> ANSI
>> code page, and for compatibility, it can't stop doing that. So the  
>> UTF-8
>> signature gives them an exit strategy.
>
> Agreed.  Having that marker at the start of the file makes interop  
> with
> other tools *much* easier.

Putting the BOM at the beginning of UTF-8 text files is not a good  
idea, it makes interop much *worse* on a unix system, not better.  
Without the BOM, most commands do the right thing with UTF-8 text.  
E.g. to concatenate two files:

$ cat file-1 file-2 > file-3

With a BOM at the beginning of the file, it won't work right. Of  
course, you could modify "cat" (and every other stream processing  
command) to know how to consume and emit BOMs, and omit the extra one  
that would show up in the middle of the stream...but even that can't  
work; what about:
$ (cat file-1; cat file-2) > file-3.

Should the shell now know that when you run multiple commands, it  
should eat the BOM emitted from the second command?

Basically, using a BOM in a utf-8 file is just not a good idea: it  
completely ruins interop with every standard unix tool.

This is not to say that Python shouldn't have a way to read a file  
with a UTF-8 BOM: it just shouldn't encourage you to *write* such files.

James


From mal at egenix.com  Fri Jan  8 22:51:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 22:51:26 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>	<4B475C72.1010207@egenix.com>
	<hi87gg$8ge$1@ger.gmane.org>
Message-ID: <4B47A8DE.1080401@egenix.com>

Tres Seaver wrote:
> M.-A. Lemburg wrote:
> 
>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
> 
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
> 
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

Sure there are non-seekable file streams, but at least when
using StringIO-type streams you don't have that restriction.

An encoding detection function would provide more use in other
cases as well, so instead of hiding away the functionality in
the open() constructor, I'm suggesting to make expose it via
the codecs module.

Applications can then use it where necessary and also provide their
own defaults, using other heuristics.

>> You'd then avoid having to stuff everything into
>> a single function call and also open up the door for more complex
>> application specific guess work or defaults.
> 
>> The whole process would then have two steps:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_file_encoding(filename)
> 
> Filename is not enough information:  or do you mean that API to actually
> open the stream?

Yes. The API would open the file, guess the encoding and then
close it again. If you don't want that to happen, you could use
the second API I mentioned below on the already open file.

Note that this function could detect a lot more encodings with
reasonably high probability than just BOM-prefixed ones,
e.g. we could also add support to detect encoding declarations
such as the ones we use in Python source files.

>>  2. open the file with the found encoding
> 
>>   f = open(filename, encoding=encoding)
> 
>> For seekable streams f, you'd have:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_stream_encoding(f)

I forgot to mention: This API needs to position the file pointer
to the start of the first data byte.

>>  2. wrap the stream with a reader for the found encoding
> 
>>   reader_class = codecs.getreader(encoding)
>>   g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From tseaver at palladion.com  Fri Jan  8 22:59:04 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:59:04 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <hi89r8$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
>>> Shouldn't this encoding guessing be a separate function that you call
>>> on either a file or a seekable stream ?
>>>
>>> After all, detecting encodings is just as useful to have for non-file
>>> streams.
>> Other stream sources typically have out-of-band ways to signal the
>> encoding:  only when reading from the filesystem do we pretty much
>> *have* to guess, and in that case the BOM / signature is the best
>> heuristic we have.  Also, some non-file streams are not seekable, and so
>> can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

If the application expects a possibly-BOM-signature-marked file, but you
pass it mismatched garbage:

  >>> f = open('some.zip', encoding='BOM")

the error handling should be the same as if you passed any other
mismatched encoding:

  >>> f = open('some.zip', encoding='UTF8')

i.e., you discover the error when you try to read from the (non)encoded
stream, not when you open it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHqpwACgkQ+gerLs4ltQ7uAACeKEc+WT4TASGcVl1Hfqe6L9La
I6EAn1pJtngtLWPdothGbYB+zUabEvTW
=TjBK
-----END PGP SIGNATURE-----



From victor.stinner at haypocalc.com  Fri Jan  8 23:10:32 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 23:10:32 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <201001082310.33029.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 22:40:47, Eric Smith a ?crit :
> >> Shouldn't this encoding guessing be a separate function that you call
> >> on either a file or a seekable stream ?
> >>
> >> After all, detecting encodings is just as useful to have for non-file
> >> streams.
> >
> > Other stream sources typically have out-of-band ways to signal the
> > encoding:  only when reading from the filesystem do we pretty much
> > *have* to guess, and in that case the BOM / signature is the best
> > heuristic we have.  Also, some non-file streams are not seekable, and so
> > can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

FYI my patch (encoding="BOM") is implemented in TextIOWrapper, and 
TextIOWrapper takes a binary stream as input, not a filename.

-- 
Victor Stinner
http://www.haypocalc.com/


From yoann.padioleau at facebook.com  Fri Jan  8 23:59:52 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Fri, 8 Jan 2010 14:59:52 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464F1C.7020404@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
	<4B464F1C.7020404@v.loewis.de>
Message-ID: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>


On Jan 7, 2010, at 1:16 PM, Martin v. L?wis wrote:

>>> astgen.py is not used to process asdl files; ast.txt lives right
>>> next to astgen.py. Instead, the asdl file is processed by
>>> Parser/asdl_c.py.
>> 
>> Yes, I know that. That's why I asked about the relation between
>> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
>> expose as a reflection mechanism things that were generated from
>> ast.txt, then there could be a mismatch. Where does ast.txt comes
>> from ? Shouldn't it be generated itself from Python.adsl ?
> 
> What you may not be aware of is that Tools/compiler (and the
> compiler package that it builds on) are both unused and unmaintained.

I see. So if people want to analyze python code they have to use
other tools (like rope?) ? or use reflection ?

> 
> If the package stops working correctly - tough luck.
> 
>> So we would have
>> 
>> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
>> containing all the UnarySub, Expression, classes that represents a
>> Python AST.
> 
> No - what actually happens in Python 3.x is this: both the compiler
> package and Tools/compiler are removed.

Ok. I will then create my own ast classes generator.

Thanks.


> 
> Regards,
> Martin



From g.brandl at gmx.net  Sat Jan  9 00:10:24 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 09 Jan 2010 00:10:24 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<4B46F54D.9090103@v.loewis.de>
	<hi878l$7os$1@ger.gmane.org>
Message-ID: <hi8e1n$sos$1@ger.gmane.org>

Am 08.01.2010 22:14, schrieb Tres Seaver:

>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly have
>> a byte order - hence I call it a signature, not the BOM. As a signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote the
>> encoding is useful and helps reducing moji-bake. This is the reason it's
>> used on Windows: notepad would normally assume that text is in the ANSI
>> code page, and for compatibility, it can't stop doing that. So the UTF-8
>> signature gives them an exit strategy.
> 
> Agreed.  Having that marker at the start of the file makes interop with
> other tools *much* easier.

Except if only 50% of the other tools support the signature.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From martin at v.loewis.de  Sat Jan  9 00:57:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 00:57:46 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>	<4B464499.4020305@v.loewis.de>	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>	<4B464F1C.7020404@v.loewis.de>
	<F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
Message-ID: <4B47C67A.3020302@v.loewis.de>

> I see. So if people want to analyze python code they have to use
> other tools (like rope?) ? or use reflection ?

Correct. One such tool might be the true Python compiler, along
with the _ast module.

Regards,
Martin


From victor.stinner at haypocalc.com  Sat Jan  9 00:59:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 00:59:00 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
Message-ID: <201001090059.00848.victor.stinner@haypocalc.com>

Hi,

Thanks for all the answers! I will try to sum up all ideas here.


(1) Change default open() behaviour or make it optional?

Guido would like to add an option and keep open() unchanged. He wrote that 
checking for BOM and using system locale are too much different to be the same 
option (encoding=None).

Antoine would like to check BOM by default, because both options (system 
locale vs checking for BOM) is the same thing.

About Antoine choice (encoding=None): which file modes would check for a BOM? 
I would like to answer only the read only mode, but then open(filename, "r") 
and open(filename, "r+") would behave differently?

  => 1 point for Guido (encoding="BOM" is more explicit)

Antoine choice has the advantage of directly support UTF-8+BOM, UTF-16 and 
UTF-32 for all applications and all modules using open(filename).

  => 1 point for Antoine


(2) Check for a BOM while reading or detect it before?

Everybody agree that checking BOM is an interesting option and should not be 
limited to open().

Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
name or a binary file-like object: it returns the encoding and seek to the 
file start or just after the BOM.

I dislike this function because it requires extra file operations (open 
(optional), read() and seek()) and it doesn't work if the file is not seekable 
(eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
avoid extra file operations.

Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
any file-like object.


(3) tell() and seek() on a text file starting with a BOM

To be consistent with Antoine example:

   >>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
   >>> f = io.TextIOWrapper(bio, encoding='utf-16')
   >>> f.read()
   'ab'
   >>> f.seek(0)
   0
   >>> f.read()
   'ab'

TextIOWrapper:

 * tell() should return zero at file start,
 * seek(0) should go be to file start,
 * and the BOM should always be "ignored".

I mean:

  with open("utf8bom.txt", encoding="BOM") as fp:
     assert fp.tell() == 0   
     text = fp.read() # no BOM here
     fp.seek(0)
     assert fp.read() == text

--

About my patch:

 - BOM check is explicit: open(filebame,  encoding="BOM")
 - tell() / seek(0) works as expected
 - BOM bytes are always skipped in read() / readlines() result

Hum, I don't know if this email can be called a sum up ;-)

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 01:09:48 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 00:09:48 +0000 (UTC)
Subject: [Python-Dev] Quick sum up about open() + BOM
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <loom.20100109T010657-648@post.gmane.org>

Hello Victor,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> (1) Change default open() behaviour or make it optional?
> 
[...]
> 
> Antoine would like to check BOM by default, because both options (system 
> locale vs checking for BOM) is the same thing.

To be clear, I am not saying it is the same thing. What I think is that it would
be a mistake to use a mildly unreliable heuristic by default (the locale +
device encoding heuristic) but refuse to trust a more reliable heuristic (the
BOM-based detection algorithm).

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Sat Jan  9 01:14:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 00:14:39 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <loom.20100109T010657-648@post.gmane.org>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<loom.20100109T010657-648@post.gmane.org>
Message-ID: <4B47CA6F.5000607@voidspace.org.uk>

On 09/01/2010 00:09, Antoine Pitrou wrote:
> Hello Victor,
>
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>    
>> (1) Change default open() behaviour or make it optional?
>>
>>      
> [...]
>    
>> Antoine would like to check BOM by default, because both options (system
>> locale vs checking for BOM) is the same thing.
>>      
> To be clear, I am not saying it is the same thing. What I think is that it would
> be a mistake to use a mildly unreliable heuristic by default (the locale +
> device encoding heuristic) but refuse to trust a more reliable heuristic (the
> BOM-based detection algorithm).
>    

I concur. On Windows both UTF-8 and signature are very common, yet the 
platform default is the truly awful CP1252.

All the best,

Michael
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From floris.bruynooghe at gmail.com  Sat Jan  9 01:26:05 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 9 Jan 2010 00:26:05 +0000
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B46F6D7.1050301@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
	<4B46F6D7.1050301@v.loewis.de>
Message-ID: <20100109002605.GB13536@laurie.devork>

On Fri, Jan 08, 2010 at 10:11:51AM +0100, "Martin v. L?wis" wrote:
> Nicholas Bastin wrote:
> > I think this problem probably needs to move over to distutils-sig, as
> > it doesn't seem to be specific to the way that Python itself uses
> > distutils.
> 
> I'm fairly skeptical that anybody on distutils SIG is interested in
> details of the Python build process...

Uh, hum.  Unfounded skepticism.  ;-)
But as said filing a bug sounds better in this case.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From v+python at g.nevcal.com  Sat Jan  9 01:47:38 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 16:47:38 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B47D22A.8070305@g.nevcal.com>

On approximately 1/8/2010 3:59 PM, came the following characters from 
the keyboard of Victor Stinner:
> Hi,
>
> Thanks for all the answers! I will try to sum up all ideas here.

One concern I have with this implementation encoding="BOM" is that if 
there is no BOM it assumes UTF-8.  That is probably a good assumption in 
some circumstances, but not in others.

* It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
(cases where the endianness is unspecified) contain a BOM.  Hence, it 
might be that someone would expect a UTF-16LE (or any of the formats 
that don't require a BOM, rather than UTF-8), but be willing to accept 
any BOM-discriminated format.

* Potentially, this could be expanded beyond the various Unicode 
encodings... one could envision that a program whose data files 
historically were in any particular national language locale, could want 
to be enhance to accept Unicode, and could declare that they will accept 
any BOM-discriminated format, but want to default, in the absence of a 
BOM, to the original national language locale that they historically 
accepted.  That would provide a migration path for their old data files.

So the point is, that it might be nice to have 
"BOM-otherEncodingForDefault" for each other encoding that Python 
supports.  Not sure that is the right API, but I think it is expressive 
enough to handle the cases above.  Whether the cases solve actual 
problems or not, I couldn't say, but they seem like reasonable cases.

It would, of course, be nicest if OS metadata had been invented way back 
when, for all OSes, such that all text files were flagged with their 
encoding... then languages could just read the encoding and do the right 
thing! But we live in the real world, instead.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From python at mrabarnett.plus.com  Sat Jan  9 02:12:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 09 Jan 2010 01:12:28 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B47D7FC.4070703@mrabarnett.plus.com>

Glenn Linderman wrote:
> On approximately 1/8/2010 3:59 PM, came the following characters from 
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if 
> there is no BOM it assumes UTF-8.  That is probably a good assumption in 
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
> (cases where the endianness is unspecified) contain a BOM.  Hence, it 
> might be that someone would expect a UTF-16LE (or any of the formats 
> that don't require a BOM, rather than UTF-8), but be willing to accept 
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode 
> encodings... one could envision that a program whose data files 
> historically were in any particular national language locale, could want 
> to be enhance to accept Unicode, and could declare that they will accept 
> any BOM-discriminated format, but want to default, in the absence of a 
> BOM, to the original national language locale that they historically 
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have 
> "BOM-otherEncodingForDefault" for each other encoding that Python 
> supports.  Not sure that is the right API, but I think it is expressive 
> enough to handle the cases above.  Whether the cases solve actual 
> problems or not, I couldn't say, but they seem like reasonable cases.
> 
> It would, of course, be nicest if OS metadata had been invented way back 
> when, for all OSes, such that all text files were flagged with their 
> encoding... then languages could just read the encoding and do the right 
> thing! But we live in the real world, instead.
> 
What about listing the possible encodings? It would try each in turn
until it found one where the BOM matched or had no BOM:

     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')

or is that taking it too far?


From martin at v.loewis.de  Sat Jan  9 02:23:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 02:23:07 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47CA6F.5000607@voidspace.org.uk>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<loom.20100109T010657-648@post.gmane.org>
	<4B47CA6F.5000607@voidspace.org.uk>
Message-ID: <4B47DA7B.6050505@v.loewis.de>

>>> Antoine would like to check BOM by default, because both options
>>> (system locale vs checking for BOM) is the same thing.
>>> 
>> To be clear, I am not saying it is the same thing. What I think is 
>> that it would be a mistake to use a mildly unreliable heuristic by
>> default (the locale + device encoding heuristic) but refuse to
>> trust a more reliable heuristic (the BOM-based detection
>> algorithm).
>> 
> 
> I concur. On Windows both UTF-8 and signature are very common, yet
> the platform default is the truly awful CP1252.

While I would support combining BOM detection in the case where a file
is opened for reading and no encoding is specified, I see two problems:
a) if a seek operations is performed before having looked at the BOM,
   no determination would have been made
b) what encoding should it use on writing?

Regards,
Martin



From v+python at g.nevcal.com  Sat Jan  9 02:49:12 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 17:49:12 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <4B47E098.6030703@g.nevcal.com>

On approximately 1/8/2010 5:12 PM, came the following characters from 
the keyboard of MRAB:
> Glenn Linderman wrote:
>> On approximately 1/8/2010 3:59 PM, came the following characters from 
>> the keyboard of Victor Stinner:
>>> Hi,
>>>
>>> Thanks for all the answers! I will try to sum up all ideas here.
>>
>> One concern I have with this implementation encoding="BOM" is that if 
>> there is no BOM it assumes UTF-8.  That is probably a good assumption 
>> in some circumstances, but not in others.
>>
>> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
>> encoded files include a BOM.  It is only required that UTF-16 and 
>> UTF-32 (cases where the endianness is unspecified) contain a BOM.  
>> Hence, it might be that someone would expect a UTF-16LE (or any of 
>> the formats that don't require a BOM, rather than UTF-8), but be 
>> willing to accept any BOM-discriminated format.
>>
>> * Potentially, this could be expanded beyond the various Unicode 
>> encodings... one could envision that a program whose data files 
>> historically were in any particular national language locale, could 
>> want to be enhance to accept Unicode, and could declare that they 
>> will accept any BOM-discriminated format, but want to default, in the 
>> absence of a BOM, to the original national language locale that they 
>> historically accepted.  That would provide a migration path for their 
>> old data files.
>>
>> So the point is, that it might be nice to have 
>> "BOM-otherEncodingForDefault" for each other encoding that Python 
>> supports.  Not sure that is the right API, but I think it is 
>> expressive enough to handle the cases above.  Whether the cases solve 
>> actual problems or not, I couldn't say, but they seem like reasonable 
>> cases.
>>
>> It would, of course, be nicest if OS metadata had been invented way 
>> back when, for all OSes, such that all text files were flagged with 
>> their encoding... then languages could just read the encoding and do 
>> the right thing! But we live in the real world, instead.
>>
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
>
>     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

That sounds very flexible -- but in net effect it would only make 
illegal a subset of the BOM-containing encodings (those not listed) 
without making legal any additional encodings other than the non-BOM 
encoding.  Whether prohibiting a subset of BOM-containing encodings is a 
useful use case, I couldn't say... but my goal would be to included as 
many different file encodings on input as possible: without a BOM, that 
is exactly 1 (unless there are other heuristics), with a BOM, it is 
1+all-BOM-containing encodings.  Your scheme would permit numbers of 
encodings accepted to vary between 1 and 1+all-BOM-containing encodings.

(I think everyone can agree there are 5 different byte sequences that 
can be called a Unicode BOM.  The likelihood of them appearing in any 
other text encoding created by mankind depends on those other encodings 
-- but it is not impossible.  It is truly up to the application to 
decide whether BOM detection could potentially conflict with files in 
some other encoding that would be acceptable to the application.)

So I think it is taking it further than I can see value in, but I'm 
willing to be convinced otherwise.  I see only a need for detecting BOM, 
and specifying a default encoding to be used if there is no BOM.  Note 
that it might be nice to have a specification for using current 
encoding=None heuristic -- perhaps encoding="BOM-None" per my originally 
proposed syntax.  But I'm still not saying that is the best syntax.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From ncoghlan at gmail.com  Sat Jan  9 04:07:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Jan 2010 13:07:12 +1000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B476196.4080409@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>
	<4B476196.4080409@mrabarnett.plus.com>
Message-ID: <4B47F2E0.7090403@gmail.com>

MRAB wrote:
> Maybe there should also be a way of determining what encoding it decided
> it was, so that you can then write a new file in that same encoding.

I thought of that question as well - the f.encoding attribute on the
opened file should be sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Sat Jan  9 06:48:36 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 06:48:36 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47E098.6030703@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com> <4B47D7FC.4070703@mrabarnett.plus.com>
	<4B47E098.6030703@g.nevcal.com>
Message-ID: <319e029f1001082148h5cee2c0bn76f70a17766ca69e@mail.gmail.com>

It seems to me that when opening a file, the following is the only
flow that makes sense for the typical opening of a file flow:

if encoding is not None:
   use encoding
elif file has BOM:
   use BOM
else:
   use system default

And hence a encoding='BOM' isn't needed there. Although I'm trying to
come up with usecases that doesn't work with this, I can't. :)

BUT

When writing things are not so easy though. Apparently some encodings
require a BOM to be written, but others do not, but allow it, and some
has no byte order mark. So there you have to be able to write the BOM,
or not. And that's either a new parameter, because you can't use
encoding='BOM' since you need to specify the encoding as well, or a
new method.

I would suggest a BOM parameter, and maybe a method as  well.

BOM=None|True|False

Where "None" means a sane default behaviour, that is write a BOM if
the encoding require it.
"True" means write a BOM if the encoding *supports* it.
"False" means Don't write a BOM even if the encoding requires it
(because I know what I'm doing)

if 'w' in mode: # But not 'r' or 'a'
    if BOM == True and encoding in (ENCODINGS THAT ALLOW BOM):
        write_bom = True
    elif BOM == False:
       write_bom = False
    elif BOM == None and encoding in (ENCODINGS THAT REQUIRE BOM):
          write_bom = True
    else:
          write_bom = False
else:
    write_bom = False

For reading this parameter could either be a noop, or possibly change
the behavior somehow, if a usecase where that makes sense can be
imagined.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From walter at livinglogic.de  Sat Jan  9 11:51:57 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 11:51:57 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B485FCD.2040901@livinglogic.de>

On 09.01.10 01:47, Glenn Linderman wrote:

> On approximately 1/8/2010 3:59 PM, came the following characters from
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.  That is probably a good assumption in
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32
> (cases where the endianness is unspecified) contain a BOM.  Hence, it
> might be that someone would expect a UTF-16LE (or any of the formats
> that don't require a BOM, rather than UTF-8), but be willing to accept
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode
> encodings... one could envision that a program whose data files
> historically were in any particular national language locale, could want
> to be enhance to accept Unicode, and could declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have
> "BOM-otherEncodingForDefault" for each other encoding that Python
> supports.  Not sure that is the right API, but I think it is expressive
> enough to handle the cases above.  Whether the cases solve actual
> problems or not, I couldn't say, but they seem like reasonable cases.

This is doable with the currect API. Simply define a codec search
function that handles all encoding names that start with "BOM-" and pass
the "otherEncodingForDefault" part along to the codec.

> It would, of course, be nicest if OS metadata had been invented way back
> when, for all OSes, such that all text files were flagged with their
> encoding... then languages could just read the encoding and do the right
> thing! But we live in the real world, instead.

Servus,
   Walter


From walter at livinglogic.de  Sat Jan  9 12:18:33 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 12:18:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081140.28124.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
Message-ID: <4B486609.2050804@livinglogic.de>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
>>> Builtin open() function is unable to open an UTF-16/32 file starting with
>>> a BOM if the encoding is not specified (raise an unicode error). For an
>>> UTF-8 file starting with a BOM, read()/readline() returns also the BOM
>>> whereas the BOM should be "ignored".
>> It depends. If you use the utf-8-sig encoding, it *will* ignore the
>> UTF-8 signature.
> 
> Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
> UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
> remove the BOM after the first read (much harder if you use a module like 
> ConfigParser or csv).
> 
>>> Since my proposition changes the result TextIOWrapper.read()/readline()
>>> for files starting with a BOM, we might introduce an option to open() to
>>> enable the new behaviour. But is it really needed to keep the backward
>>> compatibility?
>> Absolutely. And there is no need to produce a new option, but instead
>> use the existing options: define an encoding that auto-detects the
>> encoding from the family of BOMs. Maybe you call it encoding="sniff".
> 
> Good idea, I choosed open(filename, encoding="BOM").

On the surface this looks like there's an encoding named "BOM", but 
looking at your patch I found that the check is still done in 
TextIOWrapper. IMHO the best approach would to the implement a *real* 
codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
the IO library. It could even be developed as a standalone project and 
published in the Cheeseshop.

To see how something like this can be done, take a look at the UTF-16 
codec, that switches to bigendian or littleendian mode depending on the 
first read/decode call.

Servus,
    Walter







From victor.stinner at haypocalc.com  Sat Jan  9 13:37:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:37:06 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47DA7B.6050505@v.loewis.de>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47CA6F.5000607@voidspace.org.uk> <4B47DA7B.6050505@v.loewis.de>
Message-ID: <201001091337.06596.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:23:07, Martin v. L?wis a ?crit :
> While I would support combining BOM detection in the case where a file
> is opened for reading and no encoding is specified, I see two problems:
> a) if a seek operations is performed before having looked at the BOM,
>    no determination would have been made

TextIOWrapper doesn't support seek to an arbitrary byte. It uses "cookie" 
which is an opaque value. Reuse a cookie from another file or an old cookie is 
forbidden (but it doesn't raise an error). This is not specific to the BOM 
checking: the problem already exist for encodings using a BOM (eg. UTF-16).

> b) what encoding should it use on writing?

Don't change anything to writing.

With Antoince choice: open('file.txt', 'w', encoding=None) continue to use the 
actual heuristic (os.device_encoding() or system locale).

With Guido choice, encoding="BOM": it raises an error, because BOM check is 
not supported when writing into a file. How could the BOM be checked when 
creating a new (empty) file!?

-- 
Victor Stinner
http://www.haypocalc.com/


From mal at egenix.com  Sat Jan  9 13:45:58 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 09 Jan 2010 13:45:58 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B487A86.4020603@egenix.com>

Victor Stinner wrote:
> (2) Check for a BOM while reading or detect it before?
> 
> Everybody agree that checking BOM is an interesting option and should not be 
> limited to open().
> 
> Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
> name or a binary file-like object: it returns the encoding and seek to the 
> file start or just after the BOM.
> 
> I dislike this function because it requires extra file operations (open 
> (optional), read() and seek()) and it doesn't work if the file is not seekable 
> (eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
> avoid extra file operations.
> 
> Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
> any file-like object.

Yes, but the implementation is limited to just BOM checking
and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

With a codecs module function we could easily extend the
encoding detection to more file types, e.g. XML files,
Python source code files, etc. that use other mechanisms
for defining the encoding.

BTW: I haven't looked at your implementation, but what happens
when your BOM check fails ? Will the implementation add the
already read bytes back to a buffer ?

This rollback action is the only reason for needing a
seekable stream in codecs.guess_stream_encoding().

Another point to consider:

AFAIK, we currently have a moratorium on changes to Python
builtins. How does that match up with the proposed changes ?

Using a new codec like Walter suggested would move the
implementation into the stdlib for which doesn't the
moratorium doesn't apply.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From victor.stinner at haypocalc.com  Sat Jan  9 13:54:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:54:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <201001091354.17239.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 01:47:38, vous avez ?crit :
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.

If no BOM is found, it fallback to the current heuristic: os.device_encoding() 
or system local.

> (...) Hence, it might be that someone would expect a UTF-16LE (or any of 
> the formats that don't require a BOM, rather than UTF-8), but be willing 
> to accept any BOM-discriminated format.
> (...) declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted

You mean "if there is a BOM, use it, otherwise fallback to a specific 
charset"? How could it be declared? Maybe:

   open("file.txt", check_bom=True, encoding="UTF16-LE")
   open("file.txt", check_bom=True, encoding="latin1")

About falling back to UTF-8, it would be written:

   open("file.txt", check_bom=True, encoding="UTF-8")

As explained before, check_bom=True is only accepted for read only file mode.

Well, why not. This is a third choice for my point (1) :-) It's between Guido 
and Antoine choice, and I like it because we can fallback to UTF-8 instead of 
the dummy system locale: Windows users will be happy to be able to use UTF-8 
:-) I prefer to fallback to a fixed encoding then depending on the system 
locale.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:34:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:34:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <201001091434.17521.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:12:28, MRAB a ?crit :
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
> 
>      my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

Yes, you're taking it foo far :-) Checking BOM is reliable, whereas *guessing* 
the charset only using the byte stream can only be an heuristic. Guess a 
charset is a complex problem, they are 3rd party library to do that, like the 
chardet project.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:38:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:38:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B486609.2050804@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <201001091438.43576.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
> > Good idea, I choosed open(filename, encoding="BOM").
> 
> On the surface this looks like there's an encoding named "BOM", but
> looking at your patch I found that the check is still done in
> TextIOWrapper. IMHO the best approach would to the implement a *real*
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
> the IO library. It could even be developed as a standalone project and
> published in the Cheeseshop.

Why not, this is another solution to the point (2) (Check for a BOM while 
reading or detect it before?). Which encoding would be used if there is not 
BOM? UTF-8 sounds like a good choice.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:50:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:50:28 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B487A86.4020603@egenix.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B487A86.4020603@egenix.com>
Message-ID: <201001091450.28497.victor.stinner@haypocalc.com>

Hi,

Le samedi 09 janvier 2010 13:45:58, vous avez ?crit :
> > Note: I implemented the BOM check in TextIOWrapper; so it's already
> > usable for any file-like object.
> 
> Yes, but the implementation is limited to just BOM checking
> and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

Sure, but that's already better than no BOM check :-) It looks like many 
people would apprecite UTF-8-SIG detection, since this encoding is common on 
Windows.

> BTW: I haven't looked at your implementation, but what happens
> when your BOM check fails ? Will the implementation add the
> already read bytes back to a buffer ?

My implementation is done between buffer.read() and decoder.decode(data). If 
there is a BOM: set the encoding and remove the BOM bytes from the byte 
string. Otherwise, use another algorithm to choose the encoding and leave the 
byte string unchanged.

It can be seen as a codec: it works like UTF-16 and UTF-32 codecs ;-)

> AFAIK, we currently have a moratorium on changes to Python
> builtins. How does that match up with the proposed changes ?

Oh yes, I forgot the moratorium. In all solutions, some of them don't change 
the API. Eg. Antoine proposed to leave the API unchanged: open(file) => 
open(file) :-) I don't know if it's compatible with the moratorium or not.

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 16:05:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 15:05:45 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <loom.20100109T160248-501@post.gmane.org>

Walter D?rwald <walter <at> livinglogic.de> writes:
> 
> On the surface this looks like there's an encoding named "BOM", but 
> looking at your patch I found that the check is still done in 
> TextIOWrapper. IMHO the best approach would to the implement a *real* 
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
> the IO library. It could even be developed as a standalone project and 
> published in the Cheeseshop.

Sorry but this is missing the point. The point here is to improve the open()
function. I'm sure people who know about encodings are able to install the
chardet library or even whip up their own BOM detection routine...


Antoine.




From benjamin at python.org  Sat Jan  9 18:29:33 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 11:29:33 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>

On behalf of the Python development team, I'm gleeful to announce the second
alpha release of Python 2.7.

Python 2.7 is scheduled to be the last major version in the 2.x series.  It
includes many features that were first released in Python 3.1.  The faster io
module, the new nested with statement syntax, improved float repr, and the
memoryview object have been backported from 3.1. Other features include an
ordered dictionary implementation, unittests improvements, and support for ttk
Tile in Tkinter.  For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7 visit:

     http://www.python.org/download/releases/2.7/

Please note that this is a development release, intended as a preview of new
features for the community, and is thus not suitable for production use.

The 2.7 documentation can be found at:

     http://docs.python.org/2.7

Please consider trying Python 2.7 with your code and reporting any bugs you may
notice to:

     http://bugs.python.org


Have fun!

--
Benjamin Peterson
2.7 Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7's contributors)


From kmtracey at gmail.com  Sat Jan  9 19:48:12 2010
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 9 Jan 2010 13:48:12 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>

On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On behalf of the Python development team, I'm gleeful to announce the
> second
> alpha release of Python 2.7.
>
>
Well yay.  Django's test suite (1242 tests) runs with just one failure on
the 2.7 alpha 2 level, and that looks to be likely due to the improved
string/float rounding so not really a problem, just a difference.  That's
down from 104 failures and 40 errors with 2.7 alpha 1.

Note on the website page http://www.python.org/download/releases/2.7/ the
"Change log for this release" link is still pointing to the alpha 1
changelog.

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100109/d5c06f60/attachment-0006.htm>

From benjamin at python.org  Sat Jan  9 19:51:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 12:51:11 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <1afaf6161001091051kb1ba08q9b96094af16c12fa@mail.gmail.com>

2010/1/9 Karen Tracey <kmtracey at gmail.com>:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>
> Well yay.? Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.? That's
> down from 104 failures and 40 errors with 2.7 alpha 1.

Excellent!

>
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Thanks. I'll fix that.

>



-- 
Regards,
Benjamin


From skip at pobox.com  Sat Jan  9 21:00:44 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:00:44 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
Message-ID: <19272.57452.972234.643008@montanaro.dyndns.org>

How much of the Unladen Swallow cPickle speedups have been incorporated into
2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
two versions I currently care about at work - we will skip 2.5 entirely).
It appears some of their speedups may have already been merged to trunk, but
I'm not sure how much.  If a patch to merge this to 2.7 is already under
consideration I won't look at it, but I interpreted Collin Winter's response
to my query on the u-s mailing list that not everything has been done yet.

Thx,

Skip



From martin at v.loewis.de  Sat Jan  9 21:09:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 09 Jan 2010 21:09:27 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T160248-501@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
Message-ID: <4B48E277.9010408@v.loewis.de>

Antoine Pitrou wrote:
> Walter D?rwald <walter <at> livinglogic.de> writes:
>> On the surface this looks like there's an encoding named "BOM", but 
>> looking at your patch I found that the check is still done in 
>> TextIOWrapper. IMHO the best approach would to the implement a *real* 
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
>> the IO library. It could even be developed as a standalone project and 
>> published in the Cheeseshop.
> 
> Sorry but this is missing the point. The point here is to improve the open()
> function. I'm sure people who know about encodings are able to install the
> chardet library or even whip up their own BOM detection routine...

How does the requirement that it be implemented as a codec miss the
point?

FWIW, I agree with Walter that if it is provided through the encoding=
argument, it should be a codec. If it is built into the open function
(for whatever reason), it must be provided by some other parameter.

I do see the point that it becomes available to end users only when
released as part of Python. However, this *also* means that applications
won't be using it for another three years or so, since they'll have to
support older Python versions as well (unless it is integrated in the
case where no encoding is specified).

Regards,
Martin


From pjenvey at underboss.org  Sat Jan  9 21:09:29 2010
From: pjenvey at underboss.org (Philip Jenvey)
Date: Sat, 9 Jan 2010 12:09:29 -0800
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <19272.57452.972234.643008@montanaro.dyndns.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>


On Jan 9, 2010, at 12:00 PM, skip at pobox.com wrote:

> How much of the Unladen Swallow cPickle speedups have been incorporated into
> 2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
> two versions I currently care about at work - we will skip 2.5 entirely).
> It appears some of their speedups may have already been merged to trunk, but
> I'm not sure how much.  If a patch to merge this to 2.7 is already under
> consideration I won't look at it, but I interpreted Collin Winter's response
> to my query on the u-s mailing list that not everything has been done yet.

They've documented their upstream patches here:

http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

--
Philip Jenvey



From skip at pobox.com  Sat Jan  9 21:21:20 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:21:20 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
Message-ID: <19272.58688.173011.412712@montanaro.dyndns.org>


    Philip> They've documented their upstream patches here:

    Philip> http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

Thanks.  That will help immensely.

Skip



From solipsis at pitrou.net  Sat Jan  9 21:28:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:28:12 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
Message-ID: <loom.20100109T212555-508@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.




From solipsis at pitrou.net  Sat Jan  9 21:32:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:32:27 +0000 (UTC)
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <loom.20100109T213033-976@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> If a patch to merge this to 2.7 is already under
> consideration I won't look at it,

Why won't you look at it? :)
Actually, if these patches are to be merged someone should certainly look at
them, and do the (possibly) remaining work.

http://bugs.python.org/issue5683
http://bugs.python.org/issue5671

Thank you

Antoine.




From skip at pobox.com  Sat Jan  9 21:43:42 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:43:42 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
In-Reply-To: <loom.20100109T213033-976@post.gmane.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<loom.20100109T213033-976@post.gmane.org>
Message-ID: <19272.60030.25319.269597@montanaro.dyndns.org>

>>>>> "Antoine" == Antoine Pitrou <solipsis at pitrou.net> writes:

    Antoine> <skip <at> pobox.com> writes:
    >> 
    >> If a patch to merge this to 2.7 is already under
    >> consideration I won't look at it,

    Antoine> Why won't you look at it? :)

I meant I wouldn't look at developing one.

Skip


From regebro at gmail.com  Sat Jan  9 23:14:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 23:14:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>

On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Right. It seems like encoding=None is the right way to go there.
encoding='BOM' would probably only work if 'BOM' isn't an encoding but
a special tag, which is ugly.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Sun Jan 10 00:25:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 23:25:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
Message-ID: <4B49105E.303@voidspace.org.uk>

On 09/01/2010 22:14, Lennart Regebro wrote:
> On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>    
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
>>      
> Right. It seems like encoding=None is the right way to go there.
> encoding='BOM' would probably only work if 'BOM' isn't an encoding but
> a special tag, which is ugly.
>
>    
I would rather see it as the default behavior for open without an 
encoding specified.

I know Guido has expressed a preference against this so I won't continue 
to flog it.

The current behavior however is that we have a 'guessing' algorithm 
based on the platform default. Currently if you open a text file in read 
mode that has a UTF-8 signature, but the platform default is something 
other than UTF-8, then we open the file using what is likely to be the 
incorrect encoding. Looking for the signature seems to be better 
behaviour in that case.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Sun Jan 10 00:40:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Jan 2010 00:40:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <4B4913DF.7050801@v.loewis.de>

>> How does the requirement that it be implemented as a codec miss the
>> point?
> 
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Yes - however, Victor currently apparently *doesn't* want it to be the
default, but wants the user to specify encoding="BOM". If so, it isn't
the default, and it is easy to implement as a codec.

>> FWIW, I agree with Walter that if it is provided through the encoding=
>> argument, it should be a codec. If it is built into the open function
>> (for whatever reason), it must be provided by some other parameter.
> 
> Why not simply encoding=None?

I don't mind. Please re-read Walter's message - it only said that
*if* this is activated through encoding="BOM", *then* it must be
a codec, and could be on PyPI. I don't think Walter was talking about
the case "it is not activated through encoding='BOM'" *at all*.

> The default value should provide the most useful
> behaviour possible. Forcing users to choose between two different autodetection
> strategies (encoding=None and another one) is a little insane IMO.

That wouldn't disturb me much. There are a lot of things in that area
that are a little insane, starting with Microsoft Windows :-)

Regards,
Martin


From henning.vonbargen at arcor.de  Sun Jan 10 12:10:02 2010
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 10 Jan 2010 12:10:02 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <mailman.2987.1263079529.28904.python-dev@python.org>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
Message-ID: <4B49B58A.4040103@arcor.de>

If Python should support BOM when reading text files,
it should also be able to *write* such files.

An encoding="BOM" argument wouldn't help here, because
it does not specify which encoding to use actually:
UFT-8, UTF-16-LE or what?

That would be a point against encoding="BOM" and
pro an additional keyword argument "use_bom" or whatever
with the following values:

None: default (old) behaviour: don't handle BOM at all

True: reading: expect BOM (raising an exception if it's
                missing). The encoding argument must be None
                or it must match the encoding implied by the
                BOM
       writing: write a BOM. The encoding argument must be
                one of the UTF encodings.
False: reading: If a BOM is present, use it to determine the
                file encoding. The encoding argument must
                be None or it must match the encoding implied by
                the BOM. (*)
                Otherwise, use the encoding argument to determine
                the encoding.
        writing: do not write a BOM. Use the encoding argument.

(*) This is a question of taste. I think some people would prefer
     a fourth value "AUTO" instead, or to swap the behaviour of
     None and False.

Henning

P.S. To make things worse, I have sometimes seen XML files with a
UTF-8 BOM, but an XML encoding declaration of "iso-8859-1".
For such files, whatever you guess will be wrong anyway...


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From brett at python.org  Sun Jan 10 19:51:26 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 10:51:26 -0800
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
Message-ID: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>

Nick Coghlan thought I should forward this to python-dev so people are aware
of this change and why it occurred (plus it indirectly informs Guido I
finally finished the work).

---------- Forwarded message ----------
From: brett.cannon <python-checkins at python.org>
Date: Sat, Jan 9, 2010 at 18:56
Subject: [Python-checkins] r77402 - in python/trunk:
Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS Python/_warnings.c
To: python-checkins at python.org


Author: brett.cannon
Date: Sun Jan 10 03:56:19 2010
New Revision: 77402

Log:
DeprecationWarning is now silent by default.

This was originally suggested by Guido, discussed on the stdlib-sig mailing
list, and given the OK by Guido directly to me. What this change essentially
means is that Python has taken a policy of silencing warnings that are only
of interest to developers by default. This should prevent users from seeing
warnings which are triggered by an application being run against a new
interpreter before the app developer has a chance to update their code.

Closes issue #7319. Thanks to Antoine Pitrou, Ezio Melotti, and Brian Curtin
for helping with the issue.


Modified:
  python/trunk/Doc/library/warnings.rst
  python/trunk/Lib/test/test_ascii_formatd.py
  python/trunk/Lib/test/test_exceptions.py
  python/trunk/Lib/warnings.py
  python/trunk/Misc/NEWS
  python/trunk/Python/_warnings.c

Modified: python/trunk/Doc/library/warnings.rst
==============================================================================
--- python/trunk/Doc/library/warnings.rst       (original)
+++ python/trunk/Doc/library/warnings.rst       Sun Jan 10 03:56:19 2010
@@ -59,7 +59,7 @@
 | :exc:`UserWarning`               | The default category for :func:`warn`.
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`DeprecationWarning`        | Base category for warnings about
deprecated   |
-|                                  | features.
        |
+|                                  | features (ignored by default).
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`SyntaxWarning`             | Base category for warnings about
dubious      |
 |                                  | syntactic features.
        |
@@ -89,6 +89,9 @@
 standard warning categories.  A warning category must always be a subclass
of
 the :exc:`Warning` class.

+.. versionchanged:: 2.7
+   :exc:`DeprecationWarning` is ignored by default.
+

 .. _warning-filter:

@@ -148,14 +151,6 @@
 :mod:`warnings` module parses these when it is first imported (invalid
options
 are ignored, after printing a message to ``sys.stderr``).

-The warnings that are ignored by default may be enabled by passing
:option:`-Wd`
-to the interpreter. This enables default handling for all warnings,
including
-those that are normally ignored by default. This is particular useful for
-enabling ImportWarning when debugging problems importing a developed
package.
-ImportWarning can also be enabled explicitly in Python code using::
-
-   warnings.simplefilter('default', ImportWarning)
-

 .. _warning-suppress:

@@ -226,6 +221,37 @@
 entries from the warnings list before each new operation).


+Updating Code For New Versions of Python
+----------------------------------------
+
+Warnings that are only of interest to the developer are ignored by default.
As
+such you should make sure to test your code with typically ignored warnings
+made visible. You can do this from the command-line by passing
:option:`-Wd`
+to the interpreter (this is shorthand for :option:`-W default`).  This
enables
+default handling for all warnings, including those that are ignored by
default.
+To change what action is taken for encountered warnings you simply change
what
+argument is passed to :option:`-W`, e.g. :option:`-W error`. See the
+:option:`-W` flag for more details on what is possible.
+
+To programmatically do the same as :option:`-Wd`, use::
+
+  warnings.simplefilter('default')
+
+Make sure to execute this code as soon as possible. This prevents the
+registering of what warnings have been raised from unexpectedly influencing
how
+future warnings are treated.
+
+Having certain warnings ignored by default is done to prevent a user from
+seeing warnings that are only of interest to the developer. As you do not
+necessarily have control over what interpreter a user uses to run their
code,
+it is possible that a new version of Python will be released between your
+release cycles.  The new interpreter release could trigger new warnings in
your
+code that were not there in an older interpreter, e.g.
+:exc:`DeprecationWarning` for a module that you are using. While you as a
+developer want to be notified that your code is using a deprecated module,
to a
+user this information is essentially noise and provides no benefit to them.
+
+
 .. _warning-functions:

 Available Functions

Modified: python/trunk/Lib/test/test_ascii_formatd.py
==============================================================================
--- python/trunk/Lib/test/test_ascii_formatd.py (original)
+++ python/trunk/Lib/test/test_ascii_formatd.py Sun Jan 10 03:56:19 2010
@@ -4,6 +4,7 @@

 import unittest
 from test_support import check_warnings, run_unittest, cpython_only
+import warnings


 class FormatDeprecationTests(unittest.TestCase):
@@ -17,6 +18,7 @@
        buf = create_string_buffer(' ' * 100)

        with check_warnings() as w:
+            warnings.simplefilter('default')
            PyOS_ascii_formatd(byref(buf), sizeof(buf), '%+.10f',
                               c_double(10.0))
            self.assertEqual(buf.value, '+10.0000000000')

Modified: python/trunk/Lib/test/test_exceptions.py
==============================================================================
--- python/trunk/Lib/test/test_exceptions.py    (original)
+++ python/trunk/Lib/test/test_exceptions.py    Sun Jan 10 03:56:19 2010
@@ -309,6 +309,7 @@
        # BaseException.__init__ triggers a deprecation warning.
        exc = BaseException("foo")
        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('default')
            self.assertEquals(exc.message, "foo")
        self.assertEquals(len(w), 1)
        self.assertEquals(w[0].category, DeprecationWarning)

Modified: python/trunk/Lib/warnings.py
==============================================================================
--- python/trunk/Lib/warnings.py        (original)
+++ python/trunk/Lib/warnings.py        Sun Jan 10 03:56:19 2010
@@ -383,8 +383,8 @@
 # Module initialization
 _processoptions(sys.warnoptions)
 if not _warnings_defaults:
-    simplefilter("ignore", category=PendingDeprecationWarning, append=1)
-    simplefilter("ignore", category=ImportWarning, append=1)
+    for cls in (DeprecationWarning, PendingDeprecationWarning,
ImportWarning):
+        simplefilter("ignore", category=cls, append=True)
    bytes_warning = sys.flags.bytes_warning
    if bytes_warning > 1:
        bytes_action = "error"

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sun Jan 10 03:56:19 2010
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------

+- Issue #7319: Silence DeprecationWarning by default.
+
 - Issue #2335: Backport set literals syntax from Python 3.x.

 Library

Modified: python/trunk/Python/_warnings.c
==============================================================================
--- python/trunk/Python/_warnings.c     (original)
+++ python/trunk/Python/_warnings.c     Sun Jan 10 03:56:19 2010
@@ -85,10 +85,10 @@

    default_action = get_warnings_attr("defaultaction");
    if (default_action == NULL) {
-       if (PyErr_Occurred()) {
-           return NULL;
-       }
-       return _default_action;
+        if (PyErr_Occurred()) {
+            return NULL;
+        }
+        return _default_action;
    }

    Py_DECREF(_default_action);
@@ -202,12 +202,12 @@

    mod_str = PyString_AsString(filename);
    if (mod_str == NULL)
-           return NULL;
+        return NULL;
    len = PyString_Size(filename);
    if (len < 0)
        return NULL;
    if (len >= 3 &&
-       strncmp(mod_str + (len - 3), ".py", 3) == 0) {
+            strncmp(mod_str + (len - 3), ".py", 3) == 0) {
        module = PyString_FromStringAndSize(mod_str, len-3);
    }
    else {
@@ -251,7 +251,7 @@

    name = PyObject_GetAttrString(category, "__name__");
    if (name == NULL)  /* XXX Can an object lack a '__name__' attribute? */
-           return;
+        return;

    f_stderr = PySys_GetObject("stderr");
    if (f_stderr == NULL) {
@@ -341,7 +341,7 @@
        rc = already_warned(registry, key, 0);
        if (rc == -1)
            goto cleanup;
-       else if (rc == 1)
+        else if (rc == 1)
            goto return_none;
        /* Else this warning hasn't been generated before. */
    }
@@ -492,9 +492,9 @@
    /* Setup filename. */
    *filename = PyDict_GetItemString(globals, "__file__");
    if (*filename != NULL) {
-           Py_ssize_t len = PyString_Size(*filename);
+            Py_ssize_t len = PyString_Size(*filename);
        const char *file_str = PyString_AsString(*filename);
-           if (file_str == NULL || (len < 0 && PyErr_Occurred()))
+            if (file_str == NULL || (len < 0 && PyErr_Occurred()))
            goto handle_error;

        /* if filename.lower().endswith((".pyc", ".pyo")): */
@@ -506,10 +506,10 @@
                tolower(file_str[len-1]) == 'o'))
        {
            *filename = PyString_FromStringAndSize(file_str, len-1);
-               if (*filename == NULL)
-                       goto handle_error;
-           }
-           else
+            if (*filename == NULL)
+                goto handle_error;
+        }
+        else
            Py_INCREF(*filename);
    }
    else {
@@ -536,8 +536,8 @@
            else {
                /* embedded interpreters don't have sys.argv, see bug
#839151 */
                *filename = PyString_FromString("__main__");
-                   if (*filename == NULL)
-                       goto handle_error;
+                if (*filename == NULL)
+                    goto handle_error;
            }
        }
        if (*filename == NULL) {
@@ -839,26 +839,29 @@
 static PyObject *
 init_filters(void)
 {
-    PyObject *filters = PyList_New(3);
+    PyObject *filters = PyList_New(4);
    const char *bytes_action;
    if (filters == NULL)
        return NULL;

    PyList_SET_ITEM(filters, 0,
+                    create_filter(PyExc_DeprecationWarning, "ignore"));
+    PyList_SET_ITEM(filters, 1,
                    create_filter(PyExc_PendingDeprecationWarning,
"ignore"));
-    PyList_SET_ITEM(filters, 1, create_filter(PyExc_ImportWarning,
"ignore"));
+    PyList_SET_ITEM(filters, 2, create_filter(PyExc_ImportWarning,
"ignore"));
    if (Py_BytesWarningFlag > 1)
        bytes_action = "error";
    else if (Py_BytesWarningFlag)
        bytes_action = "default";
    else
        bytes_action = "ignore";
-    PyList_SET_ITEM(filters, 2, create_filter(PyExc_BytesWarning,
+    PyList_SET_ITEM(filters, 3, create_filter(PyExc_BytesWarning,
                    bytes_action));

    if (PyList_GET_ITEM(filters, 0) == NULL ||
        PyList_GET_ITEM(filters, 1) == NULL ||
-        PyList_GET_ITEM(filters, 2) == NULL) {
+        PyList_GET_ITEM(filters, 2) == NULL ||
+        PyList_GET_ITEM(filters, 3) == NULL) {
        Py_DECREF(filters);
        return NULL;
     }
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db6da5fd/attachment-0006.htm>

From victor.stinner at haypocalc.com  Sun Jan 10 20:22:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 10 Jan 2010 20:22:10 +0100
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
Message-ID: <201001102022.10259.victor.stinner@haypocalc.com>

Hi,

Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
> Nick Coghlan thought I should forward this to python-dev so people are
>  aware of this change and why it occurred (plus it indirectly informs Guido
>  I finally finished the work).

Thanks to copy this mail to the python-dev mailing list.

What is the recommanded way to get the previous behaviour (print deprecation 
warnings once)? Is there a command line option? Is it "-W default"?

-- 
Victor Stinner
http://www.haypocalc.com/


From benjamin at python.org  Sun Jan 10 20:23:54 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 13:23:54 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <201001102022.10259.victor.stinner@haypocalc.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
	<201001102022.10259.victor.stinner@haypocalc.com>
Message-ID: <1afaf6161001101123g366d80dbjeaa80f1a632605e2@mail.gmail.com>

2010/1/10 Victor Stinner <victor.stinner at haypocalc.com>:
> Hi,
>
> Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
>> Nick Coghlan thought I should forward this to python-dev so people are
>> ?aware of this change and why it occurred (plus it indirectly informs Guido
>> ?I finally finished the work).
>
> Thanks to copy this mail to the python-dev mailing list.
>
> What is the recommanded way to get the previous behaviour (print deprecation
> warnings once)? Is there a command line option? Is it "-W default"?

That commit conveniently adds documentation answering that question. :)



-- 
Regards,
Benjamin


From nas at arctrix.com  Sun Jan 10 20:30:09 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:30:09 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <hid9s1$i05$1@ger.gmane.org>

Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm gleeful to announce
> the second alpha release of Python 2.7.

Thanks to everyone who contributed.

> Python 2.7 is scheduled to be the last major version in the 2.x
> series.

Has this really been decided already? Maybe I missed it.  In my
opinion, it does Python's reputation harm to make such a statement.
Conservative users (probably the vast majority of Python users)
don't like to hear that software they are considering using is
nearing the end of its life.  What does it gain us to announce that
the 2.x branch is dead aside from bugfixes?

I propose that the 2.x branch be treated like 2.x.y branches: as
long as there is sufficient volunteer labour, it should continue to
live.  In order to avoid wasted development effort, it would be
prudent to announce that unless a 2.8 release manager steps up,
whatever is committed to the 2.x branch after the 2.7 release may
never get released.

Said another way, it's okay for the Python developers to decide to
abandon 2.x and put their efforts into 3.x. It's not okay for them
to prevent others from continuing to work on 2.x or to somehow make
2.x look worse so 3.x looks better. Python 3 needs to stand on its
own terms and I'm confident it can.

Best regards,

  Neil



From brett at python.org  Sun Jan 10 21:09:08 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:09:08 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>

On Sun, Jan 10, 2010 at 11:30, Neil Schemenauer <nas at arctrix.com> wrote:

> Benjamin Peterson <benjamin at python.org> wrote:
> > On behalf of the Python development team, I'm gleeful to announce
> > the second alpha release of Python 2.7.
>
> Thanks to everyone who contributed.
>
> > Python 2.7 is scheduled to be the last major version in the 2.x
> > series.
>
> Has this really been decided already? Maybe I missed it.


More or less. It was first discussed at the language summit last year and
has come up here a couple of times. If needed we can make it official in
terms of lifetime of 2.7, etc. at the language summit this year.


>  In my
> opinion, it does Python's reputation harm to make such a statement.
> Conservative users (probably the vast majority of Python users)
> don't like to hear that software they are considering using is
> nearing the end of its life.  What does it gain us to announce that
> the 2.x branch is dead aside from bugfixes?
>
> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.
>
> Said another way, it's okay for the Python developers to decide to
> abandon 2.x and put their efforts into 3.x. It's not okay for them
> to prevent others from continuing to work on 2.x or to somehow make
> 2.x look worse so 3.x looks better. Python 3 needs to stand on its
> own terms and I'm confident it can.
>

I don't think ending the 2.x series at 2.7 makes it look bad compared to
3.2; it's simply the end of a development line like any other software
project. I suspect 2.7 will have a protracted bugfix window because so much
code runs on 2.x exclusively at the moment. And if core developers want to
continue to backport fixes past two years  from release they can (or however
long we decide to officially support 2.7).

No one is saying people still can't work on the code, just that python-dev
as an entity is not going to focus its effort on the 2.x series anymore and
people should not rely upon us to continue to focus new development efforts
in that series. If there are core developers who want to continue to do
bugfix releases then that's fine and I am happy to flag patches as needing
backports and let other's do the work after the standard two year
maintenance cycle, but I know I do not want to be held accountable as a core
developer to keep the 2.x going indefinitely. Maintaining four branches is
enough of a reason in my book to not keep the 2.x series going.

If there really is an outcry on this we can re-visit the issue, but as of
right now we need to move forward at some point and 2.7 seems like that good
point.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/4bff0434/attachment-0006.htm>

From ncoghlan at gmail.com  Sun Jan 10 21:23:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 11 Jan 2010 06:23:27 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A373F.9050601@gmail.com>

Neil Schemenauer wrote:
> In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

The announcement is precisely to avoid the situation where people commit
new features to the 2.x main line of development (after the 2.7
maintenance branch is created) in the expectation that they will be
released as part of a hypothetical 2.8 release.

Whether that info needs to be in each and every 2.7 announcement...
probably not. It isn't really info for users of Python, just for
developers of Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Sun Jan 10 21:25:29 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:25:29 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
Message-ID: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>

Michael has given me the hg transition/stdlib time slot at the language
summit this year. In regards to that I plan to lead a discussion on:

* where we are at w/ the Hg transition (Dirkjan should be there and I did a
blog post on this topic recently:
http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
* argparse (PEP 389)
* brief mention on still wanting to break out the stdlib from CPython
* an official policy on extension modules? (i.e. must have a pure Python
implementation which can mean a ctypes implementation unless given an
explicit waiver)

That's everything from a stdlib perspective. I have half-baked ideas, but
nothing concrete enough to bring up unless people really want to discuss
stuff like how to potentially standardize module deprecation warnings, etc.

In terms of me personally, I do plan to bring up at some point during the
summit these points which don't squarely fit during my time slot:

* an official unofficial policy on how new proposed features should come to
us (i.e. working code to python-ideas with a shell of a PEP that includes
abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
* any changes needed to the issue tracker to help with the workflow? (stage
field seems like a failed experiment and we now have several effective
triage people who can help w/ guiding changes)

If there is something missing from the stdlib discussion that you think
should be brought up at the summit let me know. And if there is something
here you want to discuss before the summit let me know and I can start a
separate thread on it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/f16584e2/attachment-0006.htm>

From dirkjan at ochtman.nl  Sun Jan 10 22:52:00 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 10 Jan 2010 22:52:00 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:25, Brett Cannon <brett at python.org> wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)

Unfortunately my flight doesn't land until about the time the
Mercurial slot ends, so while I'll be there later on that afternoon
for sprinting or questions or anything, I won't be at the actual
Mercurial talk at the summit.

Cheers,

Dirkjan


From fuzzyman at voidspace.org.uk  Sun Jan 10 23:27:58 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 10 Jan 2010 22:27:58 +0000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
Message-ID: <4B4A546E.8010808@voidspace.org.uk>

On 10/01/2010 21:52, Dirkjan Ochtman wrote:
> On Sun, Jan 10, 2010 at 21:25, Brett Cannon<brett at python.org>  wrote:
>    
>> Michael has given me the hg transition/stdlib time slot at the language
>> summit this year. In regards to that I plan to lead a discussion on:
>> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
>> blog post on this topic recently:
>> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
>>      
> Unfortunately my flight doesn't land until about the time the
> Mercurial slot ends, so while I'll be there later on that afternoon
> for sprinting or questions or anything, I won't be at the actual
> Mercurial talk at the summit.
>
>    
We will probably be in 'open discussion' when you arrive so it will 
still be good to hear from you on the topic and there maybe questions 
that can't be answered until you arrive... :-)

Michael

> Cheers,
>
> Dirkjan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Mon Jan 11 00:02:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:02:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A5C69.7090007@v.loewis.de>

> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

I think it's more difficult than that. It also takes developer effort
to *commit* to the trunk. So if the trunk continued to live, would
it still be ok if I closed a 2.x feature request as "won't fix", or
only committed the new feature to the 3.x branch?

As for old branches: they *don't* live in the way you claim (i.e.
remain open with changes potentially just not released). Instead,
at some point, they are frozen to bug-fix only, then to security-fix
only, and then to no change at all.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 00:07:58 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:07:58 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A373F.9050601@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>
Message-ID: <4B4A5DCE.3070909@v.loewis.de>

> The announcement is precisely to avoid the situation where people commit
> new features to the 2.x main line of development (after the 2.7
> maintenance branch is created) in the expectation that they will be
> released as part of a hypothetical 2.8 release.
> 
> Whether that info needs to be in each and every 2.7 announcement...
> probably not. It isn't really info for users of Python, just for
> developers of Python.

I think the announcement is indeed also for the users, and I would
prefer it to stay in the release announcements, so that people will
know for sure (and speak up) before the 2.7 release happens.

As for decisions: I don't think there was an official BDFL pronouncent,
but I recall Guido posting a message close to that, proposing that 2.7
will be a release that will see bug fix releases for an indefinite
period of time (where indefinite != infinite). This was shortly after
him proposing that perhaps we shouldn't make a 2.7 release at all, and
stop at 2.6.

As for such a decision giving a bad light on Python: I don't think that
will be the case. Instead, I hear many users surprised for how long
we have been maintaining to parallel versions - "that must have taken
a lot of effort". So everybody will likely understand that enough is
enough.

Regards,
Martin


From benjamin at python.org  Mon Jan 11 01:07:35 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 18:07:35 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>

Consider this program:

class Descr(object):
    def __init__(self, name):
        self.name = name
    def __set__(self, instance, what):
        instance.__dict__[self.name] = what

class X(object):
    attr = Descr("attr")

x = X()
print(x.attr)
x.attr = 42
print(x.attr)

It gives in output:

<__main__.Descr object at 0x7fe1c9b28150>
42

The documentation [1] says that Descr is a data descriptor because it
defines the __set__ method. It also states that data descriptors
always override the value in the instance dictionary. So, the second
line should also be the descriptor object according to the
documentation.

My question is: Is this a doc bug or a implementation bug? If the
former, it will be the description of a data descriptor much less
consistent, since it will require that a __get__ method be present,
too. If the latter, the fix may break some programs relying on the
ability to "cache" a value in the instance dictionary.

[1] http://docs.python.org/reference/datamodel#invoking-descriptors


-- 
Regards,
Benjamin


From amauryfa at gmail.com  Mon Jan 11 01:51:09 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 11 Jan 2010 01:51:09 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>

Hi,

2010/1/11 Benjamin Peterson <benjamin at python.org>:
> Consider this program:
>
> class Descr(object):
> ? ?def __init__(self, name):
> ? ? ? ?self.name = name
> ? ?def __set__(self, instance, what):
> ? ? ? ?instance.__dict__[self.name] = what
>
> class X(object):
> ? ?attr = Descr("attr")
>
> x = X()
> print(x.attr)
> x.attr = 42
> print(x.attr)
>
> It gives in output:
>
> <__main__.Descr object at 0x7fe1c9b28150>
> 42
>
> The documentation [1] says that Descr is a data descriptor because it
> defines the __set__ method. It also states that data descriptors
> always override the value in the instance dictionary. So, the second
> line should also be the descriptor object according to the
> documentation.
>
> My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
>
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

Quoting the documentation:
"""Normally, data descriptors define both __get__() and __set__(),
while non-data descriptors have just the __get__() method.
"""
Your example is neither a data descriptor nor a non-data descriptor...

The thing that worries me a bit is the "x.attr" returning the Descr
object. Descriptors should remain at the class level, and instance
should only see values. I'd prefer an AttributeError in this case.

-- 
Amaury Forgeot d'Arc


From benjamin at python.org  Mon Jan 11 02:00:32 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 19:00:32 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
Message-ID: <1afaf6161001101700u21006708l2ef62bfa257e4ce7@mail.gmail.com>

2010/1/10 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...

See the footnote: http://docs.python.org/reference/datamodel#id7

>
> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.

Far too late for that, I'm afraid.



-- 
Regards,
Benjamin


From nas at arctrix.com  Mon Jan 11 02:44:57 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:44:57 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
Message-ID: <20100111014457.GA5289@arctrix.com>

On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> I don't think ending the 2.x series at 2.7 makes it look bad
> compared to 3.2; it's simply the end of a development line like
> any other software project. I suspect 2.7 will have a protracted
> bugfix window because so much code runs on 2.x exclusively at the
> moment.

I would guess over 99% of all Python code written doesn't run on
Python 3.  Given that, I think it is premature to close the door on
new major versions of Python 2.x. Also, we as a project should be
careful not to present the image that Python 2.x will not be
supported in the future.

> If there really is an outcry on this we can re-visit the issue,
> but as of right now we need to move forward at some point and 2.7
> seems like that good point.

I think that's bad PR.  If I had a successful product, I would not
announce its end of life just to see how many customers scream and
then decide if I should devote more resources to continue
maintaining it.

IMHO, the release notes should say something like:

     After the Python 2.7 release, the focus of Python development
     will be on Python 3.  There will continue to be maintainance
     releases of Python 2.x.


trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil


From tjreedy at udel.edu  Mon Jan 11 03:26:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Jan 2010 21:26:34 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <hie28p$joo$1@ger.gmane.org>

On 1/10/2010 8:44 PM, Neil Schemenauer wrote:
> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
>> I don't think ending the 2.x series at 2.7 makes it look bad
>> compared to 3.2; it's simply the end of a development line like
>> any other software project. I suspect 2.7 will have a protracted
>> bugfix window because so much code runs on 2.x exclusively at the
>> moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.

If the removal of old features had been done in the 2.x series, as once 
planned (Guido originally proposed removing the old meaning of int / int 
in 2.5) the same more or less would be true of 2.7. It is past time for 
other old and now duplicated features to be removed also.

   Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>
>> If there really is an outcry on this we can re-visit the issue,
>> but as of right now we need to move forward at some point and 2.7
>> seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.

Python is not being ended, but upgraded (with bloating duplications 
removed). Think of 3.1 as 2.8 with a new name.

tjr



From lrekucki at gmail.com  Mon Jan 11 04:26:48 2010
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Mon, 11 Jan 2010 04:26:48 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <dfbefb091001101926l366043f8mb95f196ba14f9c9@mail.gmail.com>

> Date: Mon, 11 Jan 2010 01:51:09 +0100
> From: "Amaury Forgeot d'Arc" <amauryfa at gmail.com>
> To: Benjamin Peterson <benjamin at python.org>
> Cc: Python Dev <python-dev at python.org>
> Subject: Re: [Python-Dev] Data descriptor doc/implementation
> ? ? ? ?inconsistency
> Message-ID:
> ? ? ? ?<e27efe131001101651y68e1da25je2a8d02f5c62ef19 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> 2010/1/11 Benjamin Peterson <benjamin at python.org>:
>> Consider this program:
>>
>> class Descr(object):
>> ? ?def __init__(self, name):
>> ? ? ? ?self.name = name
>> ? ?def __set__(self, instance, what):
>> ? ? ? ?instance.__dict__[self.name] = what
>>
>> class X(object):
>> ? ?attr = Descr("attr")
>>
>> x = X()
>> print(x.attr)
>> x.attr = 42
>> print(x.attr)
>>
>> It gives in output:
>>
>> <__main__.Descr object at 0x7fe1c9b28150>
>> 42
>>
>> The documentation [1] says that Descr is a data descriptor because it
>> defines the __set__ method. It also states that data descriptors
>> always override the value in the instance dictionary. So, the second
>> line should also be the descriptor object according to the
>> documentation.
>>
>> My question is: Is this a doc bug or a implementation bug? If the
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...
Actually, there is this footnote [1]:

""" A descriptor can define any combination of __get__(), __set__()
and __delete__(). If it does not define __get__(), then accessing the
attribute even on an instance will return the descriptor object
itself. If the descriptor defines __set__() and/or __delete__(), it is
a data descriptor; if it defines neither, it is a non-data descriptor.
"""

Which would mean Descr is actually a data descriptor without a
__get__(), so x.attr should always return the descriptor object itself
(at least in the docs).

> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.
>
> --
> Amaury Forgeot d'Arc

[1] http://docs.python.org/reference/datamodel#id7
-- 
Lukasz Rekucki


From brett at python.org  Mon Jan 11 04:46:04 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 19:46:04 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
Message-ID: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>

On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> > I don't think ending the 2.x series at 2.7 makes it look bad
> > compared to 3.2; it's simply the end of a development line like
> > any other software project. I suspect 2.7 will have a protracted
> > bugfix window because so much code runs on 2.x exclusively at the
> > moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x.


Well yeah; Python 3.0 is just over a year old with 3.1 -- the first robust
3.x release - is about six months old. From the beginning uptake was
expected to take years, not months, so saying that 3.x is not popular enough
seems premature.


> Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>

No one has said bugfixes will cease.


>
> > If there really is an outcry on this we can re-visit the issue,
> > but as of right now we need to move forward at some point and 2.7
> > seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.
>
>
I never said that I wanted to make this announcement specifically to provoke
outcries. I said if there happened to be outcries we could possibly visit
the issue again. Basically I was being considerate and trying to leave the
door open to discuss things in the future even though I don't see the
situation changing.


> IMHO, the release notes should say something like:
>
>     After the Python 2.7 release, the focus of Python development
>     will be on Python 3.  There will continue to be maintainance
>     releases of Python 2.x.
>

No because that suggests new features will be coming to 2.x which is not
going to happen. If you want to say there will be continual bugfix releases
for 2.7 as is par the course for Python and that the number of bugfix
releases might be more than normal then I am okay with that.


>
>
> trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil
>

That came and went already a couple months ago when we discussed stopping at
2.6 instead of 2.7 (see the various threads at
http://mail.python.org/pipermail/python-dev/2009-November/thread.html).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db586677/attachment-0006.htm>

From nas at arctrix.com  Mon Jan 11 05:05:07 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:05:07 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
Message-ID: <20100111040507.GA7200@arctrix.com>

On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> >     After the Python 2.7 release, the focus of Python development
> >     will be on Python 3.  There will continue to be maintainance
> >     releases of Python 2.x.
> 
> No because that suggests new features will be coming to 2.x which is not
> going to happen. If you want to say there will be continual bugfix releases
> for 2.7 as is par the course for Python and that the number of bugfix
> releases might be more than normal then I am okay with that.

Are you are saying that if someone steps up to merge the Unladen
Swallow features into a 2.8 release and someone also steps up to cut
the release that they will be prevented from doing so?

Also, are you presuming to channel the BDFL or was this dicussed
somewhere other than python-dev? Maybe I'm overreacting but I get
the feeling that the larger and less active segment of the Python
develpment team hasn't been involved in these decisions.

Best regards,

  Neil


From nas at arctrix.com  Mon Jan 11 05:17:54 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:17:54 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5C69.7090007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
Message-ID: <20100111041754.GB7200@arctrix.com>

On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
> [...] would it still be ok if I closed a 2.x feature request as
> "won't fix", or only committed the new feature to the 3.x branch?

Yes.  Non-bugfix development on 2.x would optional (i.e. done by
people who want to spend the time). Since language changes are now
out (an initiative I completely support), the majority of non-bugfix
changes would be optimizations and platform porting.

Regards,

  Neil


From brett at python.org  Mon Jan 11 06:06:15 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 21:06:15 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111040507.GA7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
Message-ID: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>

On Sun, Jan 10, 2010 at 20:05, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> > On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> > >     After the Python 2.7 release, the focus of Python development
> > >     will be on Python 3.  There will continue to be maintainance
> > >     releases of Python 2.x.
> >
> > No because that suggests new features will be coming to 2.x which is not
> > going to happen. If you want to say there will be continual bugfix
> releases
> > for 2.7 as is par the course for Python and that the number of bugfix
> > releases might be more than normal then I am okay with that.
>
> Are you are saying that if someone steps up to merge the Unladen
> Swallow features into a 2.8 release and someone also steps up to cut
> the release that they will be prevented from doing so?
>
>
I honestly don't know, but it's a possibility just like any other new
feature request. If people start taking the carrots we have added to 3.x and
backporting them to keep the 2.x series alive you are essentially making the
3.x DOA by negating its benefits which I personally don't agree with.


> Also, are you presuming to channel the BDFL or was this dicussed
> somewhere other than python-dev?


This was discussed; see the November 2009 threads. Guido actually suggested
ending the 2.x branch at 2.6 until people spoke up about the amount of stuff
already backported to 2.7 from 3.x because it was being assumed to be the
end of the series to warrant keeping it to help transition to 2.7.


> Maybe I'm overreacting but I get
> the feeling that the larger and less active segment of the Python
> develpment team hasn't been involved in these decisions.
>

This all came up in November from the 3rd through the 6th (four days) over a
ton of email traffic. This was not a snap decision but a heated discussion
where even Guido participated that culminated with the decision to stop at
2.7 for the 2.x series; this was not a smoked-filled room decision. I'm
sorry if people missed it when they weren't looking, but python-dev is
supposed to be the place to watch for this kind of stuff. I don't know how
else to bring this stuff to people's attention short of also on
python-committers, but developers are basically expected to be on both lists
so I don't see where anyone did anything wrong in terms of informing
developers.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/18c3bcd7/attachment-0006.htm>

From nas at arctrix.com  Mon Jan 11 06:27:13 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 23:27:13 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
Message-ID: <20100111052713.GA8211@arctrix.com>

On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> If people start taking the carrots we have added to 3.x and
> backporting them to keep the 2.x series alive you are essentially
> making the 3.x DOA by negating its benefits which I personally
> don't agree with.

I think we have got to the heart of our disagreement. Assume that
some superhuman takes all the backwards compatible goodies from 3.x
and merges them into 2.x. If that really would kill off Python 3
then I would proclaim it a project that deserved to die. Why should
the backwards incompatible changes be endured if they cannot justify
their existance by the benefit they provide?

I guess I have more confidence in Python 3 than you do. I don't see
why Python 2.x needs to be artificially limited so that Python 3 can
benefit.

Regards,

  Neil


From brett at python.org  Mon Jan 11 07:30:49 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 22:30:49 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
Message-ID: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:27, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> > If people start taking the carrots we have added to 3.x and
> > backporting them to keep the 2.x series alive you are essentially
> > making the 3.x DOA by negating its benefits which I personally
> > don't agree with.
>
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x. If that really would kill off Python 3
> then I would proclaim it a project that deserved to die. Why should
> the backwards incompatible changes be endured if they cannot justify
> their existance by the benefit they provide?
>
>
When I said "carrots" I meant everything that makes Python 3 the best
version of Python, including the backward-incompatible changes.

I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.
>

I don't view it as artificial but simply a focusing of the development team
on what has been determined as the future of Python by Guido and letting go
of the past.

IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev
saying "Python 3 is the future, but we are keeping the old Python 2.x around
because we don't have *that* much faith in the future we have laid out".
That's poison to Python 3 by showing a lack of confidence in the direction
that the BDFL and python-dev as a group has chosen. Now I could be wrong and
there could actually be a large number of active contributors who want to
keep the 2.x series going, but based on the discussion that occurred the
last time this came up I believe the guys who are keeping Python running are
ready to move on.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/36448e5f/attachment-0006.htm>

From regebro at gmail.com  Mon Jan 11 08:08:14 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 08:08:14 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <319e029f1001102308p5de87cber2131f3aec1abc9f0@mail.gmail.com>

On Mon, Jan 11, 2010 at 06:27, Neil Schemenauer <nas at arctrix.com> wrote:
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x.

The superhumans of core developers pretty much already have. That's
pretty much 2.7 you are talking about. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From chrism at plope.com  Mon Jan 11 08:27:03 2010
From: chrism at plope.com (Chris McDonough)
Date: Mon, 11 Jan 2010 02:27:03 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
Message-ID: <4B4AD2C7.1050703@plope.com>

Brett Cannon wrote:
> IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev 
> saying "Python 3 is the future, but we are keeping the old Python 2.x 
> around because we don't have *that* much faith in the future we have 
> laid out". That's poison to Python 3 by showing a lack of confidence in 
> the direction that the BDFL and python-dev as a group has chosen. Now I 
> could be wrong and there could actually be a large number of active 
> contributors who want to keep the 2.x series going, but based on the 
> discussion that occurred the last time this came up I believe the guys 
> who are keeping Python running are ready to move on.

I think this is mostly just a branding issue.  Once the folks who have 
historically kept Python 2 running really *do* move on, there will be other 
folks who will step up and become maintainers out of necessity: those stuck on 
old platforms, permanent stragglers, people who for whatever reason actually 
like Python 2 better, etc.  It's just going to happen, there's really nothing 
anybody can do about it.

I don't think there's anything wrong with this.  If such a group of folks wants 
to get together and create another Python 2.X release, there's nothing stopping 
them from doing so except the above branding declaration.  I'd personally not 
close the door on allowing these folks to call such an effort "Python 2".

- C



From martin at v.loewis.de  Mon Jan 11 09:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:06:16 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
	<20100111041754.GB7200@arctrix.com>
Message-ID: <4B4ADBF8.3030803@v.loewis.de>

Neil Schemenauer wrote:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time). 

I think *that* would give a very bad impression of Python. Depending
whom you deal with, the new feature you want may or may not get
added to the code base. Contributors would feel even more stranded
than they do now, where it may take several years to get a patch
reviewed, as you then could submit a patch, and pray that a comitter
reviews it who believes in future 2.x releases.

The point of setting policies is that it gives every user (contributors,
committers, and "end-user" developers) a reliable foundation for
expectations.

Regards,
Martin


From arcriley at gmail.com  Mon Jan 11 09:06:10 2010
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 11 Jan 2010 03:06:10 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AD2C7.1050703@plope.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com> 
	<4B4AD2C7.1050703@plope.com>
Message-ID: <bad82a81001110006n3cacd953g98fb25e96330ee89@mail.gmail.com>

after all these years, some people still run AmigaOS.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/4fdcbb4e/attachment-0006.htm>

From martin at v.loewis.de  Mon Jan 11 09:11:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:11:25 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <4B4ADD2D.6070906@v.loewis.de>

> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.

Why that? It is a fact: 2.x will not be supported, in some future.
Should we lie to users?

>      After the Python 2.7 release, the focus of Python development
>      will be on Python 3.  There will continue to be maintainance
>      releases of Python 2.x.

It shouldn't say that, because it wouldn't be true.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 09:18:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:18:30 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <4B4ADED6.5080207@v.loewis.de>

> I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.

Because it takes too much time. Too much of my time, but apparently
also too much of other people's time.

Of course, the less active fraction of Python contributors may not
notice, since they just chose to not contribute (which, of course,
is fine). However, asking me to work twice as much as I want to
on the project to keep two branches alive is just unfair.

This has nothing to do with pushing 3.x, but all with managing
available manpower and still providing quality software.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 09:42:51 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 09:42:51 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>

This is what it says now:

"Python 2.7 is scheduled to be the last major version in the 2.x
series before it moves into 5 years of bugfix-only mode. "

And during this discussion it has been noted that others than the core
python team might pick up Python 2 and make releases. So maybe we can
and this discussion by changing that line in future releases to be:

"Python 2.7 is scheduled to be the last major version in the 2.x
series released by the Python Software Foundation before it moves into
5 years of bugfix-only mode. "

That doesn't exclude *others* making a Python 2.8 that could include
all sorts of crazy features. :)

This is mainly just to get the pointless discussion over with. The
current Python core team don't want to make a 2.8, so that will not
happen. Someone else might, but as Chris points out, they won't step
up until they have to, which is probably at least two years from now.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 10:06:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 10:06:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
Message-ID: <4B4AEA25.7010100@v.loewis.de>

> "Python 2.7 is scheduled to be the last major version in the 2.x
> series released by the Python Software Foundation before it moves into
> 5 years of bugfix-only mode. "
> 
> That doesn't exclude *others* making a Python 2.8 that could include
> all sorts of crazy features. :)
> 
> This is mainly just to get the pointless discussion over with.

I know you are just looking for a compromise, but this shouldn't be it:
the PSF has deliberately stayed out of the actual Python engineering,
so the release that Benjamin makes is not done by the PSF (but by
Benjamin and his contributors :-).

I like the wording as it is (although I find the promise of five years
of bug fix releases a bit too strong).

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 10:19:24 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 10:19:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AEA25.7010100@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
Message-ID: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>

On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I know you are just looking for a compromise, but this shouldn't be it:
> the PSF has deliberately stayed out of the actual Python engineering,
> so the release that Benjamin makes is not done by the PSF (but by
> Benjamin and his contributors :-).

Hm. Yeah. That's right of course. I started with saying "official",
but then I thought "official according to who?". :) So I changed it to
mentioning PSF, but that doesn't work either. I guess the current
writing as as good as it gets, unless we change "scheduled" to
"expected" or something.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Mon Jan 11 10:42:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 Jan 2010 10:42:05 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A5C69.7090007@v.loewis.de> <20100111041754.GB7200@arctrix.com>
Message-ID: <hierpd$dm1$1@ger.gmane.org>

Neil Schemenauer, 11.01.2010 05:17:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time).

Note that there's also the time it takes to make a release (usually 
involving at least one beta release), plus the possibility of introducing 
bugs while adding new features or even while fixing agreed bugs. All of 
this takes time in addition to the time people might want to invest in 
'real' development on the 2.x trunk.

Stefan



From stephen at xemacs.org  Mon Jan 11 10:59:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 11 Jan 2010 18:59:57 +0900
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <8763793qsi.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Schemenauer writes:
 > On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:

 > > If people start taking the carrots we have added to 3.x and
 > > backporting them to keep the 2.x series alive you are essentially
 > > making the 3.x DOA by negating its benefits which I personally
 > > don't agree with.

Well, I think it's *worse* than that, and I don't think you really
mean "DOA", anyway.  (Feel free to correct me, of course.)

The problem I see with backporting lots of stuff, and/or adding new
features that aren't in 3.0, to 2.x is that it will make 2.x even
cruftier, when it was already crufty enough that Guido (and almost all
of python-dev) bit the bullet and said "backward compatibility is no
excuse for keeping something in 3.0".

That surely means that a lot of python-dev denizens will declare
non-support 2.x for x > 7.  It's not going to be the gradual migration
we've seen over the past few months as active people start to spend
more and more time on 3 vs. 2; it will be a watershed.  Especially if
these are new features merged from outside that the "small active
segment" doesn't know anything about.  From the users' point of view,
that amounts to a *fork*, even if it's internal and "friendly".

 > I think we have got to the heart of our disagreement. Assume that
 > some superhuman takes all the backwards compatible goodies from 3.x
 > and merges them into 2.x.

Isn't that a bit ridiculous?  I just don't see any evidence that
anything like that is going to happen.  Worse, if we *assume* it will
happen, I don't see any way to assess whether (1) Python 3 goes belly
up, (2) there's an effective fork confusing the users and draining the
energy of python-dev, or (3) everybody goes "wow!" because it's so
cool that everybody wants to keep maintaining an extra 3 branches
indefinitely.

My opinion is that given the clear direction the "small active
segment" is going, telling the users anything but what Brett proposed
is disinformation.

 > I guess I have more confidence in Python 3 than you do. I don't see
 > why Python 2.x needs to be artificially limited so that Python 3 can
 > benefit.

It's not for Python 3, which you, I, and I'm pretty sure Brett-in-his-
heart-of-hearts agree can take care of itself because it is *better*
than Python 2.  It's for Python, and for the Python community.


From walter at livinglogic.de  Mon Jan 11 11:37:56 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 11:37:56 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001091438.43576.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
Message-ID: <4B4AFF84.7070206@livinglogic.de>

On 09.01.10 14:38, Victor Stinner wrote:

> Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
>>> Good idea, I choosed open(filename, encoding="BOM").
>>
>> On the surface this looks like there's an encoding named "BOM", but
>> looking at your patch I found that the check is still done in
>> TextIOWrapper. IMHO the best approach would to the implement a *real*
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
>> the IO library. It could even be developed as a standalone project and
>> published in the Cheeseshop.
> 
> Why not, this is another solution to the point (2) (Check for a BOM while 
> reading or detect it before?). Which encoding would be used if there is not 
> BOM? UTF-8 sounds like a good choice.

UTF-8 might be a good choice, are the failback could be specified in the
encoding name, i.e.

   open("file.txt", encoding="BOM-UTF-8")

falls back to UTF-8, if there's no BOM at the start.

This could be implemented via a custom codec search function (see
http://docs.python.org/library/codecs.html#codecs.register for more info).

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 12:12:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:12:20 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4AFF84.7070206@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
Message-ID: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>

On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
> UTF-8 might be a good choice

No, fallback if there is no BOM should be the local settings, just as
fallback is today if you don't specify a codec.
I mean, what if you want to look for a BOM but fall back to something
else? How far will we go with encoding special information in the
codecs names? codec='BOM else UTF-16 else locale'? :-)

BOM is not a locale, and should not be a locale. Having a locale
called BOM is wrong per se. It should either be default to look for a
BOM when codec=None, or a special parameter. If none of these are
desired, then we need a special function that takes a filename or file
handle, and looks for a BOM and returns the codec found or None. But
I find that much less natural and obvious than checking the BOM when codec=None.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From regebro at gmail.com  Mon Jan 11 12:13:00 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:13:00 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <319e029f1001110313u1d839deodfe06a6c7ec423cf@mail.gmail.com>

On Mon, Jan 11, 2010 at 12:12, Lennart Regebro <regebro at gmail.com> wrote:
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se.

D'oh! I mean codec here obviously.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 13:29:04 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 13:29:04 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4913DF.7050801@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>
Message-ID: <4B4B1990.90705@livinglogic.de>

On 10.01.10 00:40, "Martin v. L?wis" wrote:
>>> How does the requirement that it be implemented as a codec miss the
>>> point?
>>
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
> 
> Yes - however, Victor currently apparently *doesn't* want it to be the
> default, but wants the user to specify encoding="BOM". If so, it isn't
> the default, and it is easy to implement as a codec.
> 
>>> FWIW, I agree with Walter that if it is provided through the encoding=
>>> argument, it should be a codec. If it is built into the open function
>>> (for whatever reason), it must be provided by some other parameter.
>>
>> Why not simply encoding=None?
> 
> I don't mind. Please re-read Walter's message - it only said that
> *if* this is activated through encoding="BOM", *then* it must be
> a codec, and could be on PyPI. I don't think Walter was talking about
> the case "it is not activated through encoding='BOM'" *at all*.

However if this autodetection feature is useful in other cases (no
matter how it's activated), it should be a codec, because as part of the
open() function it isn't reusable.

>> The default value should provide the most useful
>> behaviour possible. Forcing users to choose between two different autodetection
>> strategies (encoding=None and another one) is a little insane IMO.

And encoding="mbcs" is a third option on Windows.

> That wouldn't disturb me much. There are a lot of things in that area
> that are a little insane, starting with Microsoft Windows :-)

;)

Servus,
   Walter


From barry at python.org  Mon Jan 11 13:37:46 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 07:37:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5DCE.3070909@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com> <4B4A5DCE.3070909@v.loewis.de>
Message-ID: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>

On Jan 10, 2010, at 6:07 PM, Martin v. L?wis wrote:

> As for decisions: I don't think there was an official BDFL pronouncent,
> but I recall Guido posting a message close to that, proposing that 2.7
> will be a release that will see bug fix releases for an indefinite
> period of time (where indefinite != infinite). This was shortly after
> him proposing that perhaps we shouldn't make a 2.7 release at all, and
> stop at 2.6.

IMO, the focus for Python 2.7 (and beyond) must be on helping people transition to Python 3.  That should be the reason why a 2.7 is released and, what should dictate whether a 2.8 is necessary.

Maybe everything people need (except manpower and round tuits) is already there.  I was pleasantly surprised to find that Distribute supports automatically running 2to3 at install time for Python packages.  This allowed me to "port" a recent (admittedly small) package to Python 3 by making it "python2.6 -3" clean and adding a couple of attributes to my setup.py[1].  I don't know how widely that feature's known, but I think it's *huge*, and exactly what we need to be doing.  The more we can make it easy for people to port their Python 2 code to Python 3, and to support that with one code base, the quicker we'll get to a predominantly Python 3 world.

So the question we should be asking is: what's missing from Python 2.7 to help with this transition?  If we can't get it into 2.7, then why, and would pushing it back to 2.8 help at all?

-Barry

[1] modulo a bug in Distribute that caused doctest in separate files to not be used when running  under Python 3.



From solipsis at pitrou.net  Mon Jan 11 13:39:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 13:39:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>  <4B4B1990.90705@livinglogic.de>
Message-ID: <1263213574.3327.0.camel@localhost>


> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

It is reusable as part of io.TextIOWrapper, though.

Regards

Antoine.




From regebro at gmail.com  Mon Jan 11 13:45:30 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 13:45:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
Message-ID: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>

On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

But an autodetect feature is not a codec. Sure it should be reusable,
but making it a codec seems to be  a weird hack to me. And how would
you reuse it if it was a codec? A reusable autodetect feature would be
useable to detect what codec it is. A autodetect codec would not be
useful for that, as it would simply just decode.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 14:21:15 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 14:21:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>	<4B4913DF.7050801@v.loewis.de>
	<4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B25CB.5030003@livinglogic.de>

On 11.01.10 13:45, Lennart Regebro wrote:

> On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
>> However if this autodetection feature is useful in other cases (no
>> matter how it's activated), it should be a codec, because as part of the
>> open() function it isn't reusable.
> 
> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

I think we already had this discussion two years ago in the context of
XML decoding ;):

http://mail.python.org/pipermail/python-dev/2007-November/075138.html

> And how would
> you reuse it if it was a codec? A reusable autodetect feature would be
> useable to detect what codec it is. A autodetect codec would not be
> useful for that, as it would simply just decode.

I have implemented an XML codec (as part of XIST:
http://pypi.python.org/pypi/ll-xist), that can do that:

>>> from ll import xml_codec
>>> import codecs
>>> c = codecs.getincrementaldecoder("xml")()
>>> c.encoding
>>> c.decode("<?xml")
u''
>>> c.encoding
>>> c.decode(" version='1.0'")
u''
>>> c.encoding
>>> c.decode(" encoding='iso-8859-1'?>")
u"<?xml version='1.0' encoding='iso-8859-1'?>"
>>> c.encoding
'iso-8859-1'

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 14:47:12 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 14:47:12 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B25CB.5030003@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B25CB.5030003@livinglogic.de>
Message-ID: <319e029f1001110547n1d1c9831p34b0eac519d13f9d@mail.gmail.com>

On Mon, Jan 11, 2010 at 14:21, Walter D?rwald <walter at livinglogic.de> wrote:
> I think we already had this discussion two years ago in the context of
> XML decoding ;):

Yup. Ans Martins answer then is my answer now:

"> So the code is good, if it is inside an XML parser, and it's bad if it
> is inside a codec?

Exactly so. This functionality just *isn't* a codec - there is no
encoding. Instead, it is an algorithm for *detecting* an encoding."

The conclusion was that a method do autodetect encodings would be
good. I think the same conclusion applies here.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 18:16:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:16:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<4B46F67F.60604@v.loewis.de>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B5CF5.50806@v.loewis.de>

> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

Well, the existing UTF-16 codec also is an autodetect feature (to
detect the endianness), and I don't consider it a weird hack.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 18:27:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:27:01 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B5CF5.50806@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B5CF5.50806@v.loewis.de>
Message-ID: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:16, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> But an autodetect feature is not a codec. Sure it should be reusable,
>> but making it a codec seems to be ?a weird hack to me.
>
> Well, the existing UTF-16 codec also is an autodetect feature (to
> detect the endianness), and I don't consider it a weird hack.

So the BOM codec should raise a UnicodeDecodeError if there is no BOM?
Because that's what it would have to do, in that case, because it
can't fall back on anything, it has to handle and implement all
encodings that have a BOM. And is it then actually very useful? You
would have to do a try/except first with encoding='BOM' and then
encoding=None to get the fallback to the standard.


I must say that I find this whole thing pretty obvious. 'BOM' is not
an encoding. Either there should be a method to get the encoding from
the BOM, returning None of there isn't one, or open() should look at
the BOM when you pass in encoding=None. Or both.

That covers all usecases, is easy and obvious. Either open(file=foo,
encoding=None) or open(file, encoding=encoding_from_bom(file))

I can't see that open(file, encoding='BOM') has any benefit over this,
covers any extra usecase and is clearer in any way. Instead it adds
something confusing: An encoding that isn't an encoding.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From python at mrabarnett.plus.com  Mon Jan 11 18:35:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:35:35 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<201001091438.43576.victor.stinner@haypocalc.com>	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <4B4B6167.40606@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
>> UTF-8 might be a good choice
> 
> No, fallback if there is no BOM should be the local settings, just as
> fallback is today if you don't specify a codec.
> I mean, what if you want to look for a BOM but fall back to something
> else? How far will we go with encoding special information in the
> codecs names? codec='BOM else UTF-16 else locale'? :-)
> 
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se. It should either be default to look for a
> BOM when codec=None, or a special parameter. If none of these are
> desired, then we need a special function that takes a filename or file
> handle, and looks for a BOM and returns the codec found or None. But
> I find that much less natural and obvious than checking the BOM when codec=None.
> 
Or pass a function that accepts a byte stream or the first few bytes and
returns the encoding and any unused bytes (because the byte stream might
not be seekable)?

def guess_encoding(byte_stream):
     data = byte_stream.read(2)
     if data == b"\xFE\xFF":
         return "UTF-16BE", b""
     return "UTF-8", data

text_file = open(filename, encoding=guess_encoding)
...


From python at mrabarnett.plus.com  Mon Jan 11 18:46:30 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:46:30 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
Message-ID: <4B4B63F6.1020309@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>> I know you are just looking for a compromise, but this shouldn't be
>> it: the PSF has deliberately stayed out of the actual Python
>> engineering, so the release that Benjamin makes is not done by the
>> PSF (but by Benjamin and his contributors :-).
> 
> Hm. Yeah. That's right of course. I started with saying "official", 
> but then I thought "official according to who?". :)

"Official" is whatever the BDFL says it is! :-)

> So I changed it to mentioning PSF, but that doesn't work either. I
> guess the current writing as as good as it gets, unless we change
> "scheduled" to "expected" or something.
> 


From martin at v.loewis.de  Mon Jan 11 18:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:59:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>	
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>	
	<4B4B5CF5.50806@v.loewis.de>
	<319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
Message-ID: <4B4B66EC.7000203@v.loewis.de>

> I must say that I find this whole thing pretty obvious. 'BOM' is not
> an encoding.

That I certainly agree with.

> That covers all usecases, is easy and obvious. Either open(file=foo,
> encoding=None) or open(file, encoding=encoding_from_bom(file))
> 
> I can't see that open(file, encoding='BOM') has any benefit over this,

Well, it would have the advantage that Walter pointed out: you can
implement it independent of the open() implementation, and even provide
it in older versions of Python.

Regards,
Martin



From regebro at gmail.com  Mon Jan 11 18:59:52 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:59:52 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B63F6.1020309@mrabarnett.plus.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
Message-ID: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:46, MRAB <python at mrabarnett.plus.com> wrote:
> "Official" is whatever the BDFL says it is! :-)

Heh, right.

So, it should say "Guido wants 2.7 to be the last main version of
Python 2, so it probably will be. We promise to release bugfixes it
for, like, ages".

No need to be needlessly formal. :-D
-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From olemis at gmail.com  Mon Jan 11 19:58:01 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 13:58:01 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
[...]
>

I had similar issues too (please read below ;o) ...

On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>

About guessing the encoding, I experienced this issue while I was
developing a Trac plugin. What I was doing is as follows :

- I guessed the MIME type + charset encoding using Trac MIME API (it
was a CSV file encoded using UTF-16)
- I read the file using `open`
- Then wrapped the file using `codecs.EncodedFile`
- Then used `csv.reader`

... and still get the BOM in the first value of the first row in the CSV file.

{{{
#!python

>>> mimetype
'utf-16-le'
>>> ef = EncodedFile(f, 'utf-8', mimetype)
}}}

IMO I think I am +1 for leaving `open` just like it is, and use module
`codecs` to deal with encodings, but I am strongly -1 for returning
the BOM while using `EncodedFile` (mainly because encoding is
explicitly supplied in ;o)

> --Guido
>

CMIIW anyway ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:


From mal at egenix.com  Mon Jan 11 21:44:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 11 Jan 2010 21:44:34 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
Message-ID: <4B4B8DB2.1060102@egenix.com>

Olemis Lang wrote:
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>> Hi,
>>>
>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>> BOM should be "ignored".
>>>
> [...]
>>
> 
> I had similar issues too (please read below ;o) ...
> 
> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
> 
> About guessing the encoding, I experienced this issue while I was
> developing a Trac plugin. What I was doing is as follows :
> 
> - I guessed the MIME type + charset encoding using Trac MIME API (it
> was a CSV file encoded using UTF-16)
> - I read the file using `open`
> - Then wrapped the file using `codecs.EncodedFile`
> - Then used `csv.reader`
> 
> ... and still get the BOM in the first value of the first row in the CSV file.

You didn't say, but I presume that the charset guessing logic
returned either 'utf-16-le' or 'utf-16-be' - those encodings don't
remove the leading BOM. The 'utf-16' codec will remove the BOM.

> {{{
> #!python
> 
>>>> mimetype
> 'utf-16-le'
>>>> ef = EncodedFile(f, 'utf-8', mimetype)
> }}}

Same here: the UTF-8 codec will not remove the BOM, you have
to use the 'utf-8-sig' codec for that.

> IMO I think I am +1 for leaving `open` just like it is, and use module
> `codecs` to deal with encodings, but I am strongly -1 for returning
> the BOM while using `EncodedFile` (mainly because encoding is
> explicitly supplied in ;o)

Note that EncodedFile() doesn't do any fancy BOM detection or
filtering. This is the job of the codecs.

Also note that BOM removal is only valid at the beginning of
a file. All subsequent BOM-bytes have to be read as-is (they
map to a zero-width non-breaking space) - without removing them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Mon Jan 11 22:12:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 07:12:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <4B4B9447.4060101@gmail.com>

Benjamin Peterson wrote:
 > My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
> 
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

I would call it a documentation bug: by leaving out the __get__ you're
telling Python to override *just* the setting of the attribute, while
still using the normal lookup process for retrieving the attribute (i.e.
allowing the attribute to be shadowed in the instance dictionary).

Adding a "print('Descriptor Invoked')" to the __set__ method in your
example:

>>> x = X()
>>> print(x.attr)
<__main__.Descr object at 0x7f283b51ac50>
>>> x.attr = 42
Descriptor invoked
>>> print(x.attr)
42
>>> x.attr = 6*9
Descriptor invoked
>>> print(x.attr)
54

Note that the behaviour here is still different from that of a data
descriptor: with a data descriptor, once it gets shadowed in the
instance dictionary, the descriptor is ignored *completely*. The only
way to get the descriptor involved again is to eliminate the shadowing.
The non-data descriptor with only __set__ is just choosing not to
override the attribute lookup process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From olemis at gmail.com  Mon Jan 11 22:29:38 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 16:29:38 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B8DB2.1060102@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
	<4B4B8DB2.1060102@egenix.com>
Message-ID: <24ea26601001111329i7f609aa1i9f5db7eb61f1fae7@mail.gmail.com>

Probably one part of this is OT , but I think it could complement the
discussion ;o)

On Mon, Jan 11, 2010 at 3:44 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Olemis Lang wrote:
>>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>>> <victor.stinner at haypocalc.com> wrote:
>>>> Hi,
>>>>
>>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>>> BOM should be "ignored".
>>>>
>> [...]
>>>
>>
>> I had similar issues too (please read below ;o) ...
>>
>> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>>>
>>
>> About guessing the encoding, I experienced this issue while I was
>> developing a Trac plugin. What I was doing is as follows :
>>
>> - I guessed the MIME type + charset encoding using Trac MIME API (it
>> was a CSV file encoded using UTF-16)
>> - I read the file using `open`
>> - Then wrapped the file using `codecs.EncodedFile`
>> - Then used `csv.reader`
>>
>> ... and still get the BOM in the first value of the first row in the CSV file.
>
> You didn't say, but I presume that the charset guessing logic
> returned either 'utf-16-le' or 'utf-16-be'

Yes. In fact they return the full mimetype 'text/csv; charset=utf-16-le' ... ;o)

> - those encodings don't
> remove the leading BOM.

... and they should ?

> The 'utf-16' codec will remove the BOM.
>

In this particular case there's nothing I can do, I have to process
whatever charset is detected in the input ;o)

>> {{{
>> #!python
>>
>>>>> mimetype
>> 'utf-16-le'
>>>>> ef = EncodedFile(f, 'utf-8', mimetype)
>> }}}
>
> Same here: the UTF-8 codec will not remove the BOM, you have
> to use the 'utf-8-sig' codec for that.
>
>> IMO I think I am +1 for leaving `open` just like it is, and use module
>> `codecs` to deal with encodings, but I am strongly -1 for returning
>> the BOM while using `EncodedFile` (mainly because encoding is
>> explicitly supplied in ;o)
>
> Note that EncodedFile() doesn't do any fancy BOM detection or
> filtering.

... directly.

> This is the job of the codecs.
>

OK ... to come back to the scope of the subject, in the general case,
I think that BOM (if any) should be handled by codecs, and therefore
indirectly by EncodedFile . If that's a explicit way of working with
encodings I'd prefer to use that wrapper explicitly in order to
(encode | decode) the file and let the codec detect whether there's a
BOM or not and ?adjust? `tell`, `read` and everything else in that
wrapper (instead of `open`).

> Also note that BOM removal is only valid at the beginning of
> a file. All subsequent BOM-bytes have to be read as-is (they
> map to a zero-width non-breaking space) - without removing them.
>

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Test cases for custom query (i.e report 9) ... PASS (1.0.0)  -
http://simelo.hg.sourceforge.net/hgweb/simelo/trac-gviz/rev/d276011e7297


From martin at v.loewis.de  Mon Jan 11 22:42:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:42:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
Message-ID: <4B4B9B55.1010709@v.loewis.de>

> So the question we should be asking is: what's missing from Python
> 2.7 to help with this transition?

Wrt. the distribute feature you've noticed: Python 3.0 already supports
that in distutils. So from day one of Python 3.x, you could have used
that approach for porting Python code. I had been promoting it ever
since, but it's taking up only slowly, partially because it has
competition in the approaches "burn the bridges" (i.e. convert to 3.x
once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
(i.e. try to write code that works in both 2.x and 3.x unmodified).

> If we can't get it into 2.7, then
> why, and would pushing it back to 2.8 help at all?

I've done a fair bit of 3.x porting, and I'm firmly convinced that
2.x can do nothing:

a) telling people that they have to move to 2.6 first actually
   hurts migration, instead of helping, because it implies to them
   that they have to drop old versions (e.g. 2.3.) - just because
   they had *always* dropped old versions before supporting new ones.
b) IMO, people also don't gain much by first migrating to 2.6.
   In principle, it gives them the opportunity to get py3k warnings.
   However, I haven't heard a single positive report where these
   warnings have actually helped people in porting. Yours is the
   first report saying that you followed the official guideline,
   but you didn't state whether doing so actually helped (or whether
   you just ported to 2.6 because the guideline told you to).
c) whatever 2.7 does (except perhaps for the warnings), it won't
   be useful to applications, because they couldn't use it, anyway:
   they need to support 2.4 and 2.5, and it won't have any of the
   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
   might actually hurt porting: people may have to special-case
   2.7 because their work-arounds for older versions may break in 2.7
   (e.g. testing whether a name is *not* defined, when it becomes
   defined in 2.7), plus it gives them an incentive to not port
   yet until they can drop support for anything before 2.7.

Inherently, 2.8 can't improve on that.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 22:44:24 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:44:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
Message-ID: <4B4B9BB8.3070505@v.loewis.de>

> So, it should say "Guido wants 2.7 to be the last main version of
> Python 2, so it probably will be. We promise to release bugfixes it
> for, like, ages".
> 
> No need to be needlessly formal. :-D

That summarizes my understanding of what Guido said fairly well :-)

+1 for putting it into the announcements.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Mon Jan 11 23:11:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 11 Jan 2010 22:11:44 +0000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4B9447.4060101@gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com>
Message-ID: <4B4BA220.20701@voidspace.org.uk>

On 11/01/2010 21:12, Nick Coghlan wrote:
> Benjamin Peterson wrote:
>   >  My question is: Is this a doc bug or a implementation bug? If the
>    
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>>      
> [snip...]
>
> Note that the behaviour here is still different from that of a data
> descriptor: with a data descriptor, once it gets shadowed in the
> instance dictionary, the descriptor is ignored *completely*. The only
> way to get the descriptor involved again is to eliminate the shadowing.
> The non-data descriptor with only __set__ is just choosing not to
> override the attribute lookup process.
>
>    

Does that mean we need a third class of descriptors that are neither 
data descriptors nor non-data descriptors?

What should we call them: really-not-data-descriptors?

All the best,

Michael

> Cheers,
> Nick.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From guido at python.org  Mon Jan 11 23:55:01 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Jan 2010 14:55:01 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9BB8.3070505@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de> 
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com> 
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com> 
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com> 
	<4B4B9BB8.3070505@v.loewis.de>
Message-ID: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>

+1 from me. (With the caveat that "time will tell" is definitely the
ultimate answer. Plans may change unexpectedly, even though we don't
expect them to.)

PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
helpful. But I trust he has ported a lot more code to 3.x than I have
personally. Are there other experiences?

--Guido

On Mon, Jan 11, 2010 at 1:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So, it should say "Guido wants 2.7 to be the last main version of
>> Python 2, so it probably will be. We promise to release bugfixes it
>> for, like, ages".
>>
>> No need to be needlessly formal. :-D
>
> That summarizes my understanding of what Guido said fairly well :-)
>
> +1 for putting it into the announcements.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Tue Jan 12 00:16:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 00:16:47 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <4B4BB15F.5020807@v.loewis.de>

Guido van Rossum wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
> 
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. 

That's really because I haven't even attempted to use it. Instead, the
software would fall flat on its own when running the test suite in 3.x.
IOW, I don't need 2.6 to tell me what might break when I can see 3.x
actually breaking.

Regards,
Martin


From david.lyon at preisshare.net  Tue Jan 12 00:22:42 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:22:42 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>


Hi Martin,

> Of course, the less active fraction of Python contributors may not
> notice, since they just chose to not contribute (which, of course,
> is fine). However, asking me to work twice as much as I want to
> on the project to keep two branches alive is just unfair.

Totally true. Actually as an end-developer I'd say that python
2.x series from a programming perspective is quite good. It
doesn't need the addition of a lot of new features imho.

So for me, I think too much time spent there would be not
yield great benefits.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Well, we all know your work is super quality. :-) That's not
being contested.

However, Quality can be measured different ways and it can
be assessed in different ways. Quality itself is a subjective
thing.

The point I'm only making is that if a piece of software
doesn't have "new" things added over time, then users
can get a reverse impression of a lack of quality.

We've all seen where 'internal' quality can increase
and user perceptions can decrease.

It could be things like improved graphics and things readily
apparent to the user.

At the moment, I would say that the "internal" quality
of the python 2.x series is super high. "external" quality
issues such as the packaging dilemma give the user the
opposite quality experience. But people are working on
that as best they can elsewhere. I'll leave it at that.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Python 3.x needs more carrots.

>From an ordinary (perphaps ignorant) user perspective there
is nothing. Yes, we know if we actually will start
programming then we will like it more.

But my wishes to Santa Claus would be allow the free
flow of PEPs for Python 3 packaging. Even encourage
it.

As an end developer, here's what I'd like from Santa
in 2010 to get me to swap to python 3:

 * get all the packages on pypi tested for python 3

 * put a web based package manager in python 3. This
   would perhaps be based around PIP (keep many
   people happy) and would look much like the built
   in web-console that you get with a $200 router.

 * Incorporate SCM based end-user software installs
   by adding to python3-stdlib the SCM support
   packages (cvs,bzr,svn,hg). This would *really*
   help in the scientific community.

 * put a web interface on distutils also so that
   we don't have to use a command line interface.
   (I want a pic of a smiley girl to great me
    when I build something - "Are you ready
    to build now Honey?"). ok - I joke. But the
    point is made.

So, ok, maybe these things aren't about 'code'
quality. But rather user experience.

Things like these do count also as "quality"
via the technical term "perception of quality".

If the PEP process is as unblocked as the
documentation implies, implying that anybody
can contribute to Python 3. Then there shouldn't
be any issue.

David









From solipsis at pitrou.net  Tue Jan 12 00:37:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 23:37:47 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100112T003506-611@post.gmane.org>

David Lyon <david.lyon <at> preisshare.net> writes:
> 
> > This has nothing to do with pushing 3.x, but all with managing
> > available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As someone who experiences the difference almost every day, I can say 3.x
definitely has enough carrots for me. The simple fact that I will be able to
raise exceptions with non-ASCII error messages without having to workaround a
hopelessly broken conversion/reporting logic is a huge improvement. And that's
only a small part of the picture.

Regards

Antoine.




From janssen at parc.com  Tue Jan 12 00:47:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 11 Jan 2010 15:47:55 PST
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <17215.1263253675@parc.com>

> David Lyon <david.lyon <at> preisshare.net> writes:
> > 
> > > This has nothing to do with pushing 3.x, but all with managing
> > > available manpower and still providing quality software.
> > 
> > Python 3.x needs more carrots.

I'd be happy to move UpLib to Python 3, when the various packages that I
need are also on Python 3.  And that depresses me to think about.  I
need

   Medusa
   ReportLab
   PyLucene
   Email
   Vobject
   Mutagen
   Pyglet
   Hachoir
   Win32

I'm on the mailing lists for a lot of these, and the only one that I
know is thinking about Python 3 is Email.  I'd expect Win32 and PIL to
also be thinking about it, but I haven't heard anything.

Bill


From david.lyon at preisshare.net  Tue Jan 12 00:47:51 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:47:51 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <1220.218.214.45.58.1263253671.squirrel@syd-srv02.ezyreg.com>

Antoine writes:

> As someone who experiences the difference almost every day, I can say 3.x
> definitely has enough carrots for me. The simple fact that I will be able
> to
> raise exceptions with non-ASCII error messages without having to
> workaround a
> hopelessly broken conversion/reporting logic is a huge improvement. And
> that's
> only a small part of the picture.

In no way could I disagree with you.

Ascii works fine for us here - but I guess we're lucky.

Just keep pushing python 3 and have a nice day..

David





From barry at python.org  Tue Jan 12 01:11:28 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:11:28 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100111191128.39020d89@freewill>

On Jan 11, 2010, at 10:42 PM, Martin v. L?wis wrote:

>Wrt. the distribute feature you've noticed: Python 3.0 already supports
>that in distutils. So from day one of Python 3.x, you could have used
>that approach for porting Python code. I had been promoting it ever
>since, but it's taking up only slowly, partially because it has
>competition in the approaches "burn the bridges" (i.e. convert to 3.x
>once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
>(i.e. try to write code that works in both 2.x and 3.x unmodified).

Interesting.  The only reason I never used it before is because I didn't know
about it.  Am I alone?

Maybe I'm projecting my own preferences, but it seems to me that supporting
both Python 2 and 3 from the same code base would be a very powerful way to
enable a large amount of existing code to support Python 3.  I'm certainly
going to do this from now on with all the libraries I maintain.  I don't have
any cycles to write code twice and I can't abandon Python 2 yet.  I'm
skeptical that code can work unmodified in both 2 and 3 without 2to3.

As an example, the one library I've already ported used a metaclass.  I don't
see any way to specify that the metaclass should be used in a portable way.
In Python 2.6 it's:

class Foo:
    __metaclass__ = Meta

and in Python 3 it's:

class Foo(metaclass=Meta):

2to3 made that pain go away.

>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>2.x can do nothing:
>
>a) telling people that they have to move to 2.6 first actually
>   hurts migration, instead of helping, because it implies to them
>   that they have to drop old versions (e.g. 2.3.) - just because
>   they had *always* dropped old versions before supporting new ones.

Is it just an implication, or is it reality?  I have yet to try to support
older versions of Python and also support Python 3.  (The one package I've
done this with so far only supports Python 2.6.)

>b) IMO, people also don't gain much by first migrating to 2.6.
>   In principle, it gives them the opportunity to get py3k warnings.
>   However, I haven't heard a single positive report where these
>   warnings have actually helped people in porting. Yours is the
>   first report saying that you followed the official guideline,
>   but you didn't state whether doing so actually helped (or whether
>   you just ported to 2.6 because the guideline told you to).

Python 2.6 has other useful features, which I want to take advantage of, so
getting py3k warnings is a bonus.  Running 'python2.6 -3 setup.py test' over
my code did in fact help clean up a couple of problems.  Seems like a good
first step when considering Python 3 support to me.

>c) whatever 2.7 does (except perhaps for the warnings), it won't
>   be useful to applications, because they couldn't use it, anyway:
>   they need to support 2.4 and 2.5, and it won't have any of the
>   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
>   might actually hurt porting: people may have to special-case
>   2.7 because their work-arounds for older versions may break in 2.7
>   (e.g. testing whether a name is *not* defined, when it becomes
>   defined in 2.7), plus it gives them an incentive to not port
>   yet until they can drop support for anything before 2.7.
>
>Inherently, 2.8 can't improve on that.

I'm not so sure.  Yes, as a package maintainer there are older versions to
think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
released, what will be the minimum version of Python provided by most OS
vendors (where the majority of Python users probably get their 'python')?  I
guess some people will have to support everything from Python 2.3 to 2.8 but
you're talking supporting something like a spread of 7 years of Python
versions.  What other platform do you support for 7 years?  Even Ubuntu long
term support (LTS) releases only live for 5 years and even then, newer Pythons
are often back ported.  Or if not, then who cares?  You're not going to use a
newer version of a library on an LTS anyway.

I'm not necessarily saying that justifies a 2.8 release.  I'm just asking how
we can make it easier for people to port to Python 3.  The automatic 2to3 has
already made it easier for me to port one library to Python 3 because I barely
had to even think about it.  Maybe that's enough.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/840169eb/attachment-0006.pgp>

From barry at python.org  Tue Jan 12 01:12:19 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:12:19 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <20100111191219.5fdd2542@freewill>

On Jan 11, 2010, at 02:55 PM, Guido van Rossum wrote:

>PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
>helpful. But I trust he has ported a lot more code to 3.x than I have
>personally. Are there other experiences?

I've only done one small library so far, but it was helpful to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/ba78a57a/attachment-0006.pgp>

From andrew at bemusement.org  Tue Jan 12 01:07:26 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 12 Jan 2010 11:07:26 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100112000726.GO32351@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> I've done a fair bit of 3.x porting, and I'm firmly convinced that
> 2.x can do nothing:
[...]
> Inherently, 2.8 can't improve on that.

I agree that there are limitations like the ones you've listed, but I
disagree with your conclusion.  Maybe you assume that it's just as hard
to move to 2.8 (using the py3k backports not available in say 2.5) as it
is to 3.x?

But a hypothetical 2.8 would also give people a way to move closer to
py3k without giving up on using all their 2.x-only dependencies.  I
think it's much more likely that libraries like Twisted can support 2.8
in the near future than 3.x.

Then, when all of your dependencies (or viable alternatives to those
dependencies) are available for 3.x, you'll have an easier transition if
you can start from a 2.x with fewer differences in features.

Fundamentally the more 2.x can converge on 3.x, the easier it will be
for users to make the leap, because it will be a smaller leap.  The
longer the 2.x series lives, the more these newer 2.x versions like 2.7
and maybe even 2.8 will be available on common platforms for people to
depend upon as minimum versions, which means that as time goes by they
can depend on a version that's closer to 3.x.  And so again, the leap
becomes easier to make.  So to me it's pretty clear that 2.8 *can*
improve the transition path to 3.x.  It may or may not be a worthwhile
use of effort for python-dev to make 2.8, but that's different to saying
it's inherently pointless.

-Andrew.


From solipsis at pitrou.net  Tue Jan 12 01:28:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 00:28:26 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <loom.20100112T012550-387@post.gmane.org>

Andrew Bennetts <andrew <at> bemusement.org> writes:
> 
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies.  I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

I don't think a 2.8 could make you much closer to 3.x. AFAICT, every possible
way to ease transition to 3.x will already be in 2.7, or almost. But perhaps I'm
lacking imagination.

Regards

Antoine.




From brian.curtin at gmail.com  Tue Jan 12 02:57:46 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 11 Jan 2010 19:57:46 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>

On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:

> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
>
> -Brett
>
I think it would be interesting to see how people are using the tracker, or
how they want to be using it. For example, there are currently over 1500
open issues with no stage set, some of which seemingly haven't been read by
anyone at all. Would a properly set stage field save issues from falling
into a black hole?

Food for thought: according to the last tracker summary, there are over 1000
open issues with a patch, and issues stay open an average of 700 days
(median 450).

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/a3439436/attachment-0006.htm>

From jackdied at gmail.com  Tue Jan 12 04:53:24 2010
From: jackdied at gmail.com (Jack Diederich)
Date: Mon, 11 Jan 2010 22:53:24 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>

On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

[sidebar]
1) the metaclass fixer was a PITA to implement.
2) 95% of __metaclass__ definitions searchable via google code were of
the "__metaclass__ = type" variety.  The 2to3 patch exists only
because of the few other uses.
3) 100% of the module level assignments in public projects were the
"__metaclass__ = type" variety which is why there isn't a fixer for
that.  Also, a fixer would have been really, really ugly (munge every
class definition in this module because there is a top level
assignment).

-Jack


From benjamin at python.org  Tue Jan 12 05:01:40 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 Jan 2010 22:01:40 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <1afaf6161001112001t5e7bab83t7d93f33fa11ce849@mail.gmail.com>

2010/1/11 Jack Diederich <jackdied at gmail.com>:
> On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
>> As an example, the one library I've already ported used a metaclass. ?I don't
>> see any way to specify that the metaclass should be used in a portable way.
>> In Python 2.6 it's:
>>
>> class Foo:
>> ? ?__metaclass__ = Meta
>>
>> and in Python 3 it's:
>>
>> class Foo(metaclass=Meta):
>>
>> 2to3 made that pain go away.
>
> [sidebar]
> 1) the metaclass fixer was a PITA to implement.

Does this make it any less useful, though? :)



-- 
Regards,
Benjamin


From steven.bethard at gmail.com  Tue Jan 12 06:57:18 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 11 Jan 2010 21:57:18 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>

On Mon, Jan 11, 2010 at 4:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

Actually there's a solution to this one too:

    FooBase = Meta('FooBase', (), {})
    class Foo(FooBase):
        ...

That should work in Python 2.X and 3.X.

I've got argparse running on Python 2.3-3.1, and the changes were
pretty easy. You can see them all in the revision here:

    http://code.google.com/p/argparse/source/detail?r=12

I have aspirations of putting all of the tricks I learned up up on the
Wiki somewhere, but I just haven't had the time.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From regebro at gmail.com  Tue Jan 12 07:30:10 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:30:10 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>

On Mon, Jan 11, 2010 at 23:55, Guido van Rossum <guido at python.org> wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
>
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. But I trust he has ported a lot more code to 3.x than I have
> personally. Are there other experiences?

It doesn't warn for that many of the unportable problems, but I'm not
sure it can warn for them either. It's certainly helpful, just not
very much. :) I think the biggest help was added in 2.6.2, and that's
warning for old integer division. It will also warn for modules that
aren't supported anymore, if I remember correctly, and that's often
helpful. But it won't warn for real tricky problems, like binary vs
text in strings, and I don't see how it can either.

And, I just realized, it doesn't warn for you using cmp or __cmp__
either, and 2to3 won't fix that, so it should actually warn for it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:49:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:49:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:11, Barry Warsaw <barry at python.org> wrote:
> Interesting. ?The only reason I never used it before is because I didn't know
> about it. ?Am I alone?

Maybe not, but the Distribute feature is there because IMO the
distutils feature by itself isn't particularily useful. You need to
write your own distutils extensions, in practice, and they are not
trivial. Distribute has simply done it for you. :)

> I'm skeptical that code can work unmodified in both 2 and 3 without 2to3.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:53:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:53:20 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <319e029f1001112253x511f8109ja2300a022010ef99@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:07, Andrew Bennetts <andrew at bemusement.org> wrote:
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. ?I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

When 2.7 was discussed several people agreed that 2.7 should include
as much 3.x stuff as possible to ease transition. That turned out to
not be very much, so I'm not sure there is more. :)

Unless of course, 2.8 starts including more of the refactored
libraries, but that's a very minor issue in porting, so it won't help
much.

To really help, it needs to start implement things that break
backwards compatibility. That would be weird. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ncoghlan at gmail.com  Tue Jan 12 10:39:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:39:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4BA220.20701@voidspace.org.uk>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com> <4B4BA220.20701@voidspace.org.uk>
Message-ID: <4B4C435B.7080903@gmail.com>

Michael Foord wrote:
>> Note that the behaviour here is still different from that of a data
>> descriptor: with a data descriptor, once it gets shadowed in the
>> instance dictionary, the descriptor is ignored *completely*. The only
>> way to get the descriptor involved again is to eliminate the shadowing.
>> The non-data descriptor with only __set__ is just choosing not to
>> override the attribute lookup process.
>>
>>    
> 
> Does that mean we need a third class of descriptors that are neither
> data descriptors nor non-data descriptors?

Not really - leaving out __get__ when defining __set__ just creates a
data descriptor that happens to use the default attribute look-up
process rather than defining a different one.

(Note that I had the data/non-data terminology backwards in my previous
message - I tend to think of the two kinds of descriptor as enforced and
non-enforced respectively precisely because I have to think about it to
remember that "functions are non-data descriptors and properties are
data descriptors, hence non-data descriptors only define __get__ while
data descriptors define __set__". I don't find the data/non-data
terminology intuitive at all)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:44:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:44:18 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>	<4B4B9BB8.3070505@v.loewis.de>	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
	<319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
Message-ID: <4B4C4472.10104@gmail.com>

Lennart Regebro wrote:
> And, I just realized, it doesn't warn for you using cmp or __cmp__
> either, and 2to3 won't fix that, so it should actually warn for it.

I have a vague recollection that we tried to warn for that and ended up
nixing the warning due to vast swarms of false alarms (or because you
couldn't write correct 2.x code without triggering it).

I'd be happy for someone to prove my recollection wrong though (i.e. by
writing a patch that implements such warnings).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:48:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:48:57 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4C4589.70906@gmail.com>

David Lyon wrote:
>> This has nothing to do with pushing 3.x, but all with managing
>> available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As Guido has said a few times, the gains are far greater for *new*
Python developers than they are for existing ones.

Existing developers have to unlearn old habits and wait for libraries
they already use to get ported. New developers can just get started with
a much cleaner language. They don't have as rich a 3rd party ecosystem
on Python 3 as they would on Python 2.x at this point in time, but
unlike existing developers they also have the luxury of cherry-picking
just those packages that already have Python 3 support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Jan 12 11:20:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:20:27 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <hihida$unc$1@ger.gmane.org>

Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit?:
> 
> For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all.

I think most issues /have/ been read. It's just that for many of them, 
nobody is interested enough in or feels competent enough for fixing them.

> Would a properly set stage field save issues from
> falling into a black hole?

I don't think this has anything to do with properly setting the stage 
field. We just have limited time and manpower. Perhaps one of our goals 
should be to reach out more to potential contributors.

> Food for thought: according to the last tracker summary, there are over
> 1000 open issues with a patch, and issues stay open an average of 700
> days (median 450).

As for open issues with a patch, a "code review" button sending this 
patch to Rietveld (or any other similar tool) is something many of us 
have been hoping for :-) It would make reviewing easier, faster and more 
thorough (because you visualize things better than by looking at a diff).

Regards

Antoine.



From solipsis at pitrou.net  Tue Jan 12 11:36:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:36:49 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <hihjc1$45m$1@ger.gmane.org>

Le Tue, 12 Jan 2010 10:20:27 +0000, Antoine Pitrou a ?crit?:
> 
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.

Speaking of which, Steve had something to say about it:
http://holdenweb.blogspot.com/2009/11/comments-or-not-public-or-
private.html

cheers

Antoine.



From ncoghlan at gmail.com  Tue Jan 12 13:10:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 22:10:14 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <4B4C66A6.2040601@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
>> For example, there are currently over
>> 1500 open issues with no stage set, some of which seemingly haven't been
>> read by anyone at all.
> 
> I think most issues /have/ been read. It's just that for many of them, 
> nobody is interested enough in or feels competent enough for fixing them.

There are actually a whole host of reasons issues can stagnate:
- a feature request may seem reasonable (hence it doesn't get rejected
outright), but the right API may not be clear (hence it doesn't get
implemented in the near term)
- a patch may be reviewed and found to have significant defects (or
simply be overreaching the stated goal) but the original patch poster
loses interest after meeting resistance in their ambition to fix
something that is "obviously" broken
- a problem may simply be hard to fix in a backwards compatible way (or
even at all!)

Ultimately it boils down to Antoine's two categories (lack of interest
and lack of relevant expertise to make a final decision) but there are a
lot of different ways to get into those two buckets.

Because we aren't ruthless about pruning those kinds of issues out of
the tracker they're the ones that are going to accumulate over time.

I'd actually be interested to know what the issue stats are like when
RFEs are excluded and when the search is limited to the items flagged as
'easy'. If easy bug fix issues are taking a long time to get closed than
that would bother me a lot more than RFEs that sit on the tracker for years.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 12 13:14:47 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:14:47 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <20100112071447.675c8f24@freewill>

On Jan 11, 2010, at 10:53 PM, Jack Diederich wrote:

>3) 100% of the module level assignments in public projects were the
>"__metaclass__ = type" variety which is why there isn't a fixer for
>that.  Also, a fixer would have been really, really ugly (munge every
>class definition in this module because there is a top level
>assignment).

And almost certainly unnecessary.  IME, those are all to easily make bare
class definitions new style in Python 2.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/29b5ff24/attachment-0006.pgp>

From barry at python.org  Tue Jan 12 13:16:09 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:16:09 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
Message-ID: <20100112071609.1dcfffa6@freewill>

On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:

>Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
>That should work in Python 2.X and 3.X.

Ugly, but good call! :)

>I've got argparse running on Python 2.3-3.1, and the changes were
>pretty easy. You can see them all in the revision here:
>
>    http://code.google.com/p/argparse/source/detail?r=12
>
>I have aspirations of putting all of the tricks I learned up up on the
>Wiki somewhere, but I just haven't had the time.

The more resources we can provide people, both in code and in documentation,
the better.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/a2ba5b3e/attachment-0006.pgp>

From fuzzyman at voidspace.org.uk  Tue Jan 12 13:29:12 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 12:29:12 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112071609.1dcfffa6@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
	<20100112071609.1dcfffa6@freewill>
Message-ID: <4B4C6B18.7050008@voidspace.org.uk>

On 12/01/2010 12:16, Barry Warsaw wrote:
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>    
>> Actually there's a solution to this one too:
>>
>>     FooBase = Meta('FooBase', (), {})
>>     class Foo(FooBase):
>>         ...
>>
>> That should work in Python 2.X and 3.X.
>>      
> Ugly, but good call! :)
>
>    

There are all sorts of tricks. For example you can do exception handling 
that works with pre-2.6 syntax and 3.0 with a bare except and using 
sys.exc_info. It is horrible, but acceptable for short pieces of code (I 
have a couple of small modules that do this).

I haven't yet tried converting larger code-bases to Python 3, but I 
think the workflow advocated by Martin is greatly preferable to the 
hacks and tricks needed to make the same codebase run under 2 & 3.

Michael

>> I've got argparse running on Python 2.3-3.1, and the changes were
>> pretty easy. You can see them all in the revision here:
>>
>>     http://code.google.com/p/argparse/source/detail?r=12
>>
>> I have aspirations of putting all of the tricks I learned up up on the
>> Wiki somewhere, but I just haven't had the time.
>>      
> The more resources we can provide people, both in code and in documentation,
> the better.
>
> Thanks!
> -Barry
>    
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/7acd54f3/attachment-0006.htm>

From mal at egenix.com  Tue Jan 12 14:09:50 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 12 Jan 2010 14:09:50 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <4B4C749E.4040009@egenix.com>

Brett Cannon wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> 
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> * argparse (PEP 389)
> * brief mention on still wanting to break out the stdlib from CPython
> * an official policy on extension modules? (i.e. must have a pure Python
> implementation which can mean a ctypes implementation unless given an
> explicit waiver)
> 
> That's everything from a stdlib perspective. I have half-baked ideas, but
> nothing concrete enough to bring up unless people really want to discuss
> stuff like how to potentially standardize module deprecation warnings, etc.
> 
> In terms of me personally, I do plan to bring up at some point during the
> summit these points which don't squarely fit during my time slot:
> 
> * an official unofficial policy on how new proposed features should come to
> us (i.e. working code to python-ideas with a shell of a PEP that includes
> abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
> 
> If there is something missing from the stdlib discussion that you think
> should be brought up at the summit let me know. And if there is something
> here you want to discuss before the summit let me know and I can start a
> separate thread on it.

Could you please put these things and the results up on the Python
wiki ?!

We're going to have a language summit at EuroPython this year
as well and may want to continue/extend the discussion based
on what you're doing at PyCon.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From brian.curtin at gmail.com  Tue Jan 12 15:33:06 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 12 Jan 2010 08:33:06 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <cf9f31f21001120633n6460b55as6064228cce41c949@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:20, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
> >
> > For example, there are currently over
> > 1500 open issues with no stage set, some of which seemingly haven't been
> > read by anyone at all.
>
> I think most issues /have/ been read. It's just that for many of them,
> nobody is interested enough in or feels competent enough for fixing them.
>
> > Would a properly set stage field save issues from
> > falling into a black hole?
>
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.


Agreed, I didn't mean to place blame on the stage field, I just ran with how
I view that field since it was mentioned. When I'm thinking in "code-writing
developer mode" (tm), I'm more likely to go to issues which have a stage so
I know what I'm going into -- what needs to be worked on. When I'm in
project cleanup mode, I go by stageless issues -- what is necessary for this
to begin or end work. Maybe others work differently...software projects take
all kinds :-)

Anyways, sorry for the off-topic. If this is a summit worthy discussion,
cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/cf466340/attachment-0006.htm>

From rdmurray at bitdance.com  Tue Jan 12 17:12:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 11:12:28 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C66A6.2040601@gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org> <4B4C66A6.2040601@gmail.com>
Message-ID: <20100112161228.300EA1D688D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 22:10:14 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There are actually a whole host of reasons issues can stagnate:
> - a feature request may seem reasonable (hence it doesn't get rejected
> outright), but the right API may not be clear (hence it doesn't get
> implemented in the near term)
> - a patch may be reviewed and found to have significant defects (or
> simply be overreaching the stated goal) but the original patch poster
> loses interest after meeting resistance in their ambition to fix
> something that is "obviously" broken
> - a problem may simply be hard to fix in a backwards compatible way (or
> even at all!)

I would add:

- a patch is reviewed but the reviewer requests a second opinion before
  committing, which never arrives, and the original reviewer forgets
  about the issue.

I have occasionally observed apparently moribund issues spring to life
and get resolved when a triage person does something as simple as updating
the releases to which an issue applies.

> Ultimately it boils down to Antoine's two categories (lack of interest
> and lack of relevant expertise to make a final decision) but there are a
> lot of different ways to get into those two buckets.

There's a third bucket here: lack of time.  Sometimes there is interest
and expertise, but no time.  Worse, sometimes we end up waiting on the
person with the expertise and interest but no time, when someone else
with somewhat less expertise but more time should just go ahead and do
the commit.  (*That* one should be fixable.)

> Because we aren't ruthless about pruning those kinds of issues out of
> the tracker they're the ones that are going to accumulate over time.

Another other category that hangs around in the tracker is items for
which there simply isn't a consensus.  I suppose those could be brought
to python-dev for a final decision if someone wanted to tackle that task.

And I don't think it is a lack of ruthlessness.  I think it is the current
community consensus that we leave these issues open in the tracker in
case someone does come along and want to deal with them. (*)

> I'd actually be interested to know what the issue stats are like when
> RFEs are excluded and when the search is limited to the items flagged as
> 'easy'. If easy bug fix issues are taking a long time to get closed than
> that would bother me a lot more than RFEs that sit on the tracker for
> years.

I'd also be interested to know what the average and median lifetime
of closed bug reports is.  Not the metrics for open issues, but the
metrics for closed ones.  My theory is that we close a lot of bugs fairly
promptly, but the ones in the above categories make the average age of
*open* issues high.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls

(*) For example, I'm currently going through all the open issues
relating to the email package, and some of them are pretty darn
old, yet still have useful content.


From brett at python.org  Tue Jan 12 18:40:13 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:40:13 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C749E.4040009@egenix.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<4B4C749E.4040009@egenix.com>
Message-ID: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>

On Tue, Jan 12, 2010 at 05:09, M.-A. Lemburg <mal at egenix.com> wrote:

> Brett Cannon wrote:
> > Michael has given me the hg transition/stdlib time slot at the language
> > summit this year. In regards to that I plan to lead a discussion on:
> >
> > * where we are at w/ the Hg transition (Dirkjan should be there and I did
> a
> > blog post on this topic recently:
> > http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> > * argparse (PEP 389)
> > * brief mention on still wanting to break out the stdlib from CPython
> > * an official policy on extension modules? (i.e. must have a pure Python
> > implementation which can mean a ctypes implementation unless given an
> > explicit waiver)
> >
> > That's everything from a stdlib perspective. I have half-baked ideas, but
> > nothing concrete enough to bring up unless people really want to discuss
> > stuff like how to potentially standardize module deprecation warnings,
> etc.
> >
> > In terms of me personally, I do plan to bring up at some point during the
> > summit these points which don't squarely fit during my time slot:
> >
> > * an official unofficial policy on how new proposed features should come
> to
> > us (i.e. working code to python-ideas with a shell of a PEP that includes
> > abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> > * any changes needed to the issue tracker to help with the workflow?
> (stage
> > field seems like a failed experiment and we now have several effective
> > triage people who can help w/ guiding changes)
> >
> > If there is something missing from the stdlib discussion that you think
> > should be brought up at the summit let me know. And if there is something
> > here you want to discuss before the summit let me know and I can start a
> > separate thread on it.
>
> Could you please put these things and the results up on the Python
> wiki ?!
>
> We're going to have a language summit at EuroPython this year
> as well and may want to continue/extend the discussion based
> on what you're doing at PyCon.
>
>
I expect there will be at least summary emails on what gets discussed. There
is also a chance that it will be videotaped.

-Brett


> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 12 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/04964610/attachment-0006.htm>

From brett at python.org  Tue Jan 12 18:47:50 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:47:50 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>

On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
>
>>  * any changes needed to the issue tracker to help with the workflow?
>> (stage field seems like a failed experiment and we now have several
>> effective triage people who can help w/ guiding changes)
>>
>> -Brett
>>
> I think it would be interesting to see how people are using the tracker, or
> how they want to be using it. For example, there are currently over 1500
> open issues with no stage set, some of which seemingly haven't been read by
> anyone at all. Would a properly set stage field save issues from falling
> into a black hole?
>
>
"Using" is the key word there. I know this thread went on about how bugs
tend to end up being left open, but what I was proposing to talk about was
whether there is any shift desired by the seasoned tracker users to help in
their work. For instance, the Stage field was meant to help, but I don't
think it is really getting used much, which makes it at best just a UI junk
and at worst something to confuse new users. So I just wanted to discuss
things as a group to see if we could streamline some things, add others,
etc.

At worst I will try to grab people like Mark D., R. David, Antoine, Georg,
etc. at the summit (not sure exactly which the heavy bug closers will be
there), take them aside, and flat-out ask what they need to make their jobs
easier.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/3f808530/attachment-0006.htm>

From brett at python.org  Tue Jan 12 19:29:06 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:29:06 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C6B18.7050008@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com> 
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org> 
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill> 
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com> 
	<20100112071609.1dcfffa6@freewill> <4B4C6B18.7050008@voidspace.org.uk>
Message-ID: <bbaeab101001121029v28a863ccg8213ed494f15160c@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 12/01/2010 12:16, Barry Warsaw wrote:
>
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>
>
>  Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
> That should work in Python 2.X and 3.X.
>
>
>  Ugly, but good call! :)
>
>
>
>
> There are all sorts of tricks. For example you can do exception handling
> that works with pre-2.6 syntax and 3.0 with a bare except and using
> sys.exc_info. It is horrible, but acceptable for short pieces of code (I
> have a couple of small modules that do this).
>
> I haven't yet tried converting larger code-bases to Python 3, but I think
> the workflow advocated by Martin is greatly preferable to the hacks and
> tricks needed to make the same codebase run under 2 & 3.
>
>
In other words we need to pull together a HOWTO for Python source like the
one for extension modules that Benjamin wrote and make it rather prominently
linked from the Python 3 documentation index page.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/86cc3a21/attachment-0006.htm>

From rdmurray at bitdance.com  Tue Jan 12 19:31:23 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 13:31:23 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
Message-ID: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> >>  * any changes needed to the issue tracker to help with the workflow?
> >> (stage field seems like a failed experiment and we now have several
> >> effective triage people who can help w/ guiding changes)
> >>
> > I think it would be interesting to see how people are using the tracker, or
> > how they want to be using it. For example, there are currently over 1500
> > open issues with no stage set, some of which seemingly haven't been read by
> > anyone at all. Would a properly set stage field save issues from falling
> > into a black hole?
> >
> "Using" is the key word there. I know this thread went on about how bugs
> tend to end up being left open, but what I was proposing to talk about was
> whether there is any shift desired by the seasoned tracker users to help in
> their work. For instance, the Stage field was meant to help, but I don't
> think it is really getting used much, which makes it at best just a UI junk
> and at worst something to confuse new users. So I just wanted to discuss
> things as a group to see if we could streamline some things, add others,
> etc.

Well, I for one find the stage field useful.  It reminds me at a glance
where the issue is at in the workflow (and 'not set' is a valid place
in the workflow that an issue sometimes stays in for a while for reason
other than not having been triaged yet).  I suspect that if we discussed
it as a group (we who make the most changes on the tracker) we could
improve it, and the interface in general.

By the way, you could talk to people who aren't going to be at the
summit on #python-dev; I think all the currently tracker-active people
hang out there on a regular basis.

I'll have to give some thought to what changes/improvements might be
most useful, now that I've been doing this for a while.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Tue Jan 12 19:34:02 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:34:02 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com> 
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com> 
	<20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
Message-ID: <bbaeab101001121034m6de86385u4e0e72301e404a59@mail.gmail.com>

On Tue, Jan 12, 2010 at 10:31, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> > On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> > >>  * any changes needed to the issue tracker to help with the workflow?
> > >> (stage field seems like a failed experiment and we now have several
> > >> effective triage people who can help w/ guiding changes)
> > >>
> > > I think it would be interesting to see how people are using the
> tracker, or
> > > how they want to be using it. For example, there are currently over
> 1500
> > > open issues with no stage set, some of which seemingly haven't been
> read by
> > > anyone at all. Would a properly set stage field save issues from
> falling
> > > into a black hole?
> > >
> > "Using" is the key word there. I know this thread went on about how bugs
> > tend to end up being left open, but what I was proposing to talk about
> was
> > whether there is any shift desired by the seasoned tracker users to help
> in
> > their work. For instance, the Stage field was meant to help, but I don't
> > think it is really getting used much, which makes it at best just a UI
> junk
> > and at worst something to confuse new users. So I just wanted to discuss
> > things as a group to see if we could streamline some things, add others,
> > etc.
>
> Well, I for one find the stage field useful.  It reminds me at a glance
> where the issue is at in the workflow (and 'not set' is a valid place
> in the workflow that an issue sometimes stays in for a while for reason
> other than not having been triaged yet).  I suspect that if we discussed
> it as a group (we who make the most changes on the tracker) we could
> improve it, and the interface in general.
>
> By the way, you could talk to people who aren't going to be at the
> summit on #python-dev; I think all the currently tracker-active people
> hang out there on a regular basis.
>
>
Sounds reasonable.


> I'll have to give some thought to what changes/improvements might be
> most useful, now that I've been doing this for a while.


How about everyone who is active on bugs.python.org give it some thought and
we can try to have a discussion at some point in the relatively near future.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/26a09128/attachment-0006.htm>

From ncoghlan at gmail.com  Tue Jan 12 21:58:49 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 06:58:49 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<4B4C749E.4040009@egenix.com>
	<bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
Message-ID: <4B4CE289.6040709@gmail.com>

Brett Cannon wrote:
> I expect there will be at least summary emails on what gets discussed.
> There is also a chance that it will be videotaped.

The Wiki makes for a better summary archive though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Tue Jan 12 22:53:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:53:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
Message-ID: <4B4CEF3D.8000107@v.loewis.de>

>> a) telling people that they have to move to 2.6 first actually
>>   hurts migration, instead of helping, because it implies to them
>>   that they have to drop old versions (e.g. 2.3.) - just because
>>   they had *always* dropped old versions before supporting new ones.
> 
> Is it just an implication, or is it reality?

That's only the implication. However, this was precisely the dialogue
when talking to Django. If you start with "start supporting 2.6", the
immediate response, without listening further, was, "ok, wait until we
drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).

Then explain it to the individual you are talking to, wait for the next
developer of the project step along, and see how he brings up the
very same line of thinking (supporting new versions == dropping support
for old versions).

I think only part of that comes from the maintenance burden. The other
part is that they *want* to drop support for old versions, so that they
can eventually start using new features (e.g. generator expressions).
So they welcome the requirement to support new versions as an excuse
to drop old ones ("it is obvious that you have to drop 2.3 to support
3.2"). However, their users then won't let them drop old versions.


> 
>> b) IMO, people also don't gain much by first migrating to 2.6.
>>   In principle, it gives them the opportunity to get py3k warnings.
>>   However, I haven't heard a single positive report where these
>>   warnings have actually helped people in porting. Yours is the
>>   first report saying that you followed the official guideline,
>>   but you didn't state whether doing so actually helped (or whether
>>   you just ported to 2.6 because the guideline told you to).
> 
> Python 2.6 has other useful features, which I want to take advantage of

I think you are a minority with that, being able to actually use the 2.6
features already. Many projects can't, as they have to support at least
2.4 still (so the with statement is right out).

>> Inherently, 2.8 can't improve on that.
> 
> I'm not so sure.  Yes, as a package maintainer there are older versions to
> think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
> released, what will be the minimum version of Python provided by most OS
> vendors (where the majority of Python users probably get their 'python')?

"Current" Linux distributions will have 2.6 then. "Old" installations
will have 2.4.

> I
> guess some people will have to support everything from Python 2.3 to 2.8 but
> you're talking supporting something like a spread of 7 years of Python
> versions.  What other platform do you support for 7 years?

I think 2.3 will really be gone by the time 2.8 might get released. Even
with 2.7, you'd end up with a span of seven years, though.

Python had been supporting Windows 95 for more than 7 years (I think
rather 9 or 10), likewise Windows 3.1 before that. Python 2.7 will
likely still support Windows 2000, which then will be ten years old.

Solaris support will probably go back to Solaris 2.6, which will be
13 years old when Python 2.7 gets released.

It's only the Linux (and OS X) releases that move so quickly.

Regards,
Martin


From martin at v.loewis.de  Tue Jan 12 22:56:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:56:55 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
Message-ID: <4B4CF027.4080007@v.loewis.de>

> Maybe not, but the Distribute feature is there because IMO the
> distutils feature by itself isn't particularily useful. You need to
> write your own distutils extensions, in practice, and they are not
> trivial.

I wouldn't say that. My Django port works with bare distutils (as
does Django itself), and it works fine.

That distribute had to redo it is only because setuptools *replaces*
the build_py command, as does the 2to3 support in distutils. So only
if you have a different build_py already, you can't use what is in
distutils.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Tue Jan 12 23:02:34 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:02:34 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CEF3D.8000107@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>
	<4B4CEF3D.8000107@v.loewis.de>
Message-ID: <4B4CF17A.1000503@voidspace.org.uk>

On 12/01/2010 21:53, "Martin v. L?wis" wrote:
>>> a) telling people that they have to move to 2.6 first actually
>>>    hurts migration, instead of helping, because it implies to them
>>>    that they have to drop old versions (e.g. 2.3.) - just because
>>>    they had *always* dropped old versions before supporting new ones.
>>>        
>> Is it just an implication, or is it reality?
>>      
> That's only the implication. However, this was precisely the dialogue
> when talking to Django. If you start with "start supporting 2.6", the
> immediate response, without listening further, was, "ok, wait until we
> drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).
>
> Then explain it to the individual you are talking to, wait for the next
> developer of the project step along, and see how he brings up the
> very same line of thinking (supporting new versions == dropping support
> for old versions).
>
> I think only part of that comes from the maintenance burden. The other
> part is that they *want* to drop support for old versions, so that they
> can eventually start using new features (e.g. generator expressions).
> So they welcome the requirement to support new versions as an excuse
> to drop old ones ("it is obvious that you have to drop 2.3 to support
> 3.2"). However, their users then won't let them drop old versions.
>
>
>    

I agree with Martin that the *perception* is that to use Python 2.6 to 
help you port to Python 3 you have to be willing to drop support for 
earlier versions of Python.

>>      
>>> b) IMO, people also don't gain much by first migrating to 2.6.
>>>    In principle, it gives them the opportunity to get py3k warnings.
>>>    However, I haven't heard a single positive report where these
>>>    warnings have actually helped people in porting. Yours is the
>>>    first report saying that you followed the official guideline,
>>>    but you didn't state whether doing so actually helped (or whether
>>>    you just ported to 2.6 because the guideline told you to).
>>>        
>> Python 2.6 has other useful features, which I want to take advantage of
>>      
> I think you are a minority with that, being able to actually use the 2.6
> features already. Many projects can't, as they have to support at least
> 2.4 still (so the with statement is right out).
>
>    
Well, the IronPython community has almost completely moved over to 
IronPython 2.6. :-)

We tend to ship our Python runtime with our applications though. As it 
happens I'm now working with CPython on the server (Python 2.5) and 
IronPython in the browser where we are using 2.6. The new property 
feature is nice, as is having with without a __future__ import.

All the best,


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From regebro at gmail.com  Tue Jan 12 23:03:41 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 23:03:41 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF027.4080007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
	<4B4CF027.4080007@v.loewis.de>
Message-ID: <319e029f1001121403x14ef7ec8x729fb80fa67feaef@mail.gmail.com>

On Tue, Jan 12, 2010 at 22:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Maybe not, but the Distribute feature is there because IMO the
>> distutils feature by itself isn't particularily useful. You need to
>> write your own distutils extensions, in practice, and they are not
>> trivial.
>
> I wouldn't say that. My Django port works with bare distutils (as
> does Django itself), and it works fine.
>
> That distribute had to redo it is only because setuptools *replaces*
> the build_py command, as does the 2to3 support in distutils. So only
> if you have a different build_py already, you can't use what is in
> distutils.

Yeah, you are right, I misremembered. The actual additional feature is
the support for the test command. Testing under Python 2 and 3 without
it is annoying.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan 12 23:04:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:04:37 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <4B4CF1F5.4050403@v.loewis.de>

> [...]
>> I've done a fair bit of 3.x porting, and I'm firmly convinced that
>> 2.x can do nothing:
> [...]
>> Inherently, 2.8 can't improve on that.
> 
> I agree that there are limitations like the ones you've listed, but I
> disagree with your conclusion.  Maybe you assume that it's just as hard
> to move to 2.8 (using the py3k backports not available in say 2.5) as it
> is to 3.x?

Not at all, no. I'd rather expect that code that runs on 2.7 will run
on 2.8 unmodified.

> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. 

How so? If they use anything that is new in 2.8, they *will* need to
drop support for anything before it, no???

> I think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
that help Twisted in moving to 3.2?

> Then, when all of your dependencies (or viable alternatives to those
> dependencies) are available for 3.x, you'll have an easier transition if
> you can start from a 2.x with fewer differences in features.

But you won't *have* fewer differences. Just because your code runs
on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
for that). This doesn't get you any closer - you can't use any of
the 2.8 features as long as you have to support older versions of
Python.

> Fundamentally the more 2.x can converge on 3.x, the easier it will be
> for users to make the leap, because it will be a smaller leap.

No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
likely won't.

> The
> longer the 2.x series lives, the more these newer 2.x versions like 2.7
> and maybe even 2.8 will be available on common platforms for people to
> depend upon as minimum versions, which means that as time goes by they
> can depend on a version that's closer to 3.x.

No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
released in 2010. Then, in 2015, it may be that everybody has migrated
to 2.7, which then is a common platform.

If you release 2.8 in 2012, then, in 2015, people will be split between
2.7 and 2.8, and so there won't be a common platform before 2017.

So stopping 2.x development *earlier* will also give us a common
platform earlier.

Regareds,
Martin


From martin at v.loewis.de  Tue Jan 12 23:07:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:07:02 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <4B4CF286.5040002@v.loewis.de>

> I think it would be interesting to see how people are using the tracker,
> or how they want to be using it. For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all. Would a properly set stage field save issues from
> falling into a black hole?

I personally don't think this would be the case.

What would help is people who actually *work* on the issues, rather than
merely reading them. However, this being open source, there is no way to
force it (unless you create an incentive, such as the five-for-one
deal).

Regards,
Martin


From python at mrabarnett.plus.com  Tue Jan 12 23:10:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 12 Jan 2010 22:10:28 +0000
Subject: [Python-Dev] regex module
Message-ID: <4B4CF354.2050603@mrabarnett.plus.com>

Hi all,

I'm back on the regex module after doing other things and I'd like your
opinion on a number of matters:

Firstly, the current re module has a bug whereby it doesn't split on
zero-width matches. The BDFL has said that this behaviour should be
retained by default in case any existing software depends on it. My
question is: should my regex module still do this for Python 3?
Speaking personally, I'd like it to behave correctly, and Python 3 is
the version where backwards-compatibility is allowed to be broken.

Secondly, Python 2 is reaching the end of the line and Python 3 is the
future. Should I still release a version that works with Python 2? I'm
thinking that it could be confusing if new regex module did zero-width
splits correctly in Python 3 but not in Python 2. And also, should I
release it only for Python 3 as a 'carrot'?

Finally, the module allows some extra backslash escapes, eg \g<name>, in
the pattern. Should it treat ill-formed escapes, eg \g, as it would have
treated them in the re module?

Thanks


From michael at voidspace.org.uk  Tue Jan 12 23:46:49 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:46:49 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
Message-ID: <4B4CFBD9.1090009@voidspace.org.uk>

I presume the email below is about the Windows binary. Does the AMD64 
release work on intel 64bit and can we make the wording clearer on the 
download page?

The current description is " Windows AMD64 binary".

All the best,

Michael

-------- Original Message --------
Subject: 	Download Page - AMD64
Date: 	Fri, 11 Dec 2009 04:03:16 -0600
From: 	Thomas Brownback <thomas.brownback at gmail.com>
To: 	webmaster at python.org



Should I use the AMD64 version of Python on an Intel 64 chip? I know 
those 64-bit implementations are very similar, but are they similar 
enough that your AMD64 will work on Intel?

If so, perhaps a note should be placed on that page to help avoid 
confusion. Explicit being better than implicit and all.

Thanks for your time,
Thomas

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/80d6b539/attachment-0006.htm>

From andrew at bemusement.org  Tue Jan 12 23:49:56 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Wed, 13 Jan 2010 09:49:56 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100112224956.GC28576@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > But a hypothetical 2.8 would also give people a way to move closer to
> > py3k without giving up on using all their 2.x-only dependencies. 
> 
> How so? If they use anything that is new in 2.8, they *will* need to
> drop support for anything before it, no???
> 
> > I think it's much more likely that libraries like Twisted can support 2.8
> > in the near future than 3.x.
> 
> Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
> that help Twisted in moving to 3.2?

I'm not talking about Twisted moving to 3.x (FWIW, I think the only
movement there so far is some patches for some -3 warnings).  The
situation I'm describing is a project X that:

  (a) has 2.x-only dependencies, and
  (b) would like to be as close as possible to 3.x (because they like
      the new features and/or want to be as ready as possible to jump
      when (a) is fixed).

So just because project X depends on e.g. Twisted, and that Twisted in
turn still supports 2.4, doesn't mean that X cannot move to 2.8, and
doesn't mean it would get no benefit from doing so.

[...]
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.

But this is my point.  I think they would as an intermediate step to
jumping to 3.x (which also requires dropping 2.5, after all!), if for
some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

-Andrew.



From tjreedy at udel.edu  Tue Jan 12 23:51:31 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 17:51:31 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <hiiudf$2uv$1@ger.gmane.org>

On 1/12/2010 5:04 PM, "Martin v. L?wis" wrote:

> But you won't *have* fewer differences. Just because your code runs
> on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
> for that). This doesn't get you any closer - you can't use any of
> the 2.8 features as long as you have to support older versions of
> Python.
>
>> Fundamentally the more 2.x can converge on 3.x, the easier it will be
>> for users to make the leap, because it will be a smaller leap.
>
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.
>
>> The
>> longer the 2.x series lives, the more these newer 2.x versions like 2.7
>> and maybe even 2.8 will be available on common platforms for people to
>> depend upon as minimum versions, which means that as time goes by they
>> can depend on a version that's closer to 3.x.
>
> No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
> released in 2010. Then, in 2015, it may be that everybody has migrated
> to 2.7, which then is a common platform.
>
> If you release 2.8 in 2012, then, in 2015, people will be split between
> 2.7 and 2.8, and so there won't be a common platform before 2017.

Just like people today may need to work with both 2.5 and 2.6, or 
privately backport 2.6 bugfixes to 2.5.

> So stopping 2.x development *earlier* will also give us a common
> platform earlier.

With years of bug fixes and hence high quality.




From tjreedy at udel.edu  Wed Jan 13 00:05:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 18:05:12 -0500
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <hiiv75$5md$1@ger.gmane.org>

On 1/12/2010 5:10 PM, MRAB wrote:
> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.

Are you writing a new module with a new name? If so, do you expect it to 
replace or augment re? (This is the same question as for optparse vs. 
argparse, which I understand to not yet be decided.)
>
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?

2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
2.7 stdlib is already out. A new engine should get some community 
testing before going in the stdlib. Even 3.2 beta is not that far off 
(8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?

> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?

What does re do with analogous cases?

Terry Jan Reedy



From david.lyon at preisshare.net  Wed Jan 13 00:20:54 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 13 Jan 2010 10:20:54 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C4589.70906@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<4B4C4589.70906@gmail.com>
Message-ID: <1333.218.214.45.58.1263338454.squirrel@syd-srv02.ezyreg.com>

> Nick wrote:
>>> This has nothing to do with pushing 3.x, but all with managing
>>> available manpower and still providing quality software.
>>
>> Python 3.x needs more carrots.
>
> As Guido has said a few times, the gains are far greater for *new*
> Python developers than they are for existing ones.

Well both you and Guido are most likely 100% correct. :-)

> They don't have as rich a 3rd party ecosystem
> on Python 3 as they would on Python 2.x at this point in time, but
> unlike existing developers they also have the luxury of cherry-picking
> just those packages that already have Python 3 support.

Most likely. I wouldn't want to say anything to discourage
people from going to python 3. In other languages, I have
much experience of making the jump to 'a whole new world'.

It's unsettling at first, but after that, you suddenly
find the new model has a better turbo than the last and
you're at the next corner faster than you can think. So
it's all good.

But I still maintain Python 3.0 needs more carrots. For example,
if mercurial or any other cool lib gets added to python 3 (and
I can name a few that should go in python 3) then they should
be added to python 3 and not to python 2.x

They would serve as good carrots.

Make fresh the python 3 stdlib and preserve the python 2.x stdlib.

I really think we are somewhat short on resources to do
what Guido has asked about bringing python up to CPAN level
with respect to packages.

We're starting a long way back with horses and swords and
trying to catch a well fed and greased modern machine..

I'm sure we can get a modern package testbot operational
for python.

But I wish those who were complaining about the packaging
problem so much could throw some money at the problem
instead of moaning about it. As is done on other open-source
projects. Some organisation would be beneficial.

Finding funds so that a small group of people could
work on the problem would be a great boost to forward
progress. I just think python packaging needs six
months of that to repair the years and years of neglect
and stagnation. Even if it is only beer and bus fare
money. It just needs a temporary shot of adrenalin in
the arm.. so to speak..

Regards

David











From martin at v.loewis.de  Wed Jan 13 00:28:14 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 00:28:14 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D058E.4030404@v.loewis.de>

> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?

"intel 64bit" is as clear as mud. It could mean the "Intel 64"
architecture, or it could mean the "IA-64" architecture, both
are 64-bit architectures with processors manufactured by Intel.
The former is officially called AMD64 (not by Intel, but
officially), and our AMD64 binaries run on such processors.
The latter is also called "Itanium", and we stopped producing
binaries for that architecture a while ago; the AMD64 binaries
will *not* run on it.

The wording could probably be changed to match the reader's
expectation more, most likely, it would also get more incorrect
in the process. To make it correct and explicit, an entire
paragraph educating users about 64-bit architectures and that
there are many of them and that they are mutually incompatible
might be required.

Most likely, Thomas' processor implements the AMD64 architecture,
even though it is manufactured by Intel. A short sentence that
he would probably understand (given that he used "Intel 64", not
"64bit") is:

"""The binaries for AMD64 will also work on processors that implement
the Intel 64 architecture (formerly EM64T), i.e. the architecture that
Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
They will not work on Intel Itanium Processors (formerly IA-64)."""

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 00:30:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 00:30:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112224956.GC28576@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>	<4B4CF1F5.4050403@v.loewis.de>
	<20100112224956.GC28576@steerpike.home.puzzling.org>
Message-ID: <4B4D0613.1010503@v.loewis.de>

> I'm not talking about Twisted moving to 3.x (FWIW, I think the only
> movement there so far is some patches for some -3 warnings).  The
> situation I'm describing is a project X that:
> 
>   (a) has 2.x-only dependencies, and
>   (b) would like to be as close as possible to 3.x (because they like
>       the new features and/or want to be as ready as possible to jump
>       when (a) is fixed).

This assumes that jumping to 3.x is easier if you are closer to it.
Please trust me that this is not the case.

>> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
>> likely won't.
> 
> But this is my point.  I think they would as an intermediate step to
> jumping to 3.x (which also requires dropping 2.5, after all!), if for
> some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

No, and no. No: it's not an intermediate step, and no, supporting 3.x
does *not* require dropping 2.5.

Regards,
Martin



From fuzzyman at voidspace.org.uk  Wed Jan 13 00:40:35 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:40:35 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D058E.4030404@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
Message-ID: <4B4D0873.5070006@voidspace.org.uk>

On 12/01/2010 23:28, "Martin v. L?wis" wrote:
> [snip...]
> """The binaries for AMD64 will also work on processors that implement
> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
> They will not work on Intel Itanium Processors (formerly IA-64)."""
>
>    

To those not intimately aware of the history and present of 64 bit 
architecture this sentence would be a great addition - thanks. I'm 
adding it now as a footnote.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From lists at cheimes.de  Wed Jan 13 00:41:04 2010
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 13 Jan 2010 00:41:04 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D0890.2030505@cheimes.de>

Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64 
> release work on intel 64bit and can we make the wording clearer on the 
> download page?
> 
> The current description is " Windows AMD64 binary".

The installer works on all AMD64 compatible Intel CPUs. *scnr*

As you most likely know all modern Intel 64bit CPUs are based on AMD's
x86-64 design. Only the Itanium family is based on the other Intel 64bit
design: IA-64. The name AMD64 was chosen because most Linux and BSD
distributions call it so. The name 'AMD64' has caused confusion in the
past because users thought that the installer works only on AMD CPUs.

How about:

 * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
X86-64 binary -- does not include source)

instead of:

 * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
not include source)

?

Christia


From fuzzyman at voidspace.org.uk  Wed Jan 13 00:55:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:55:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0873.5070006@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
	<4B4D0873.5070006@voidspace.org.uk>
Message-ID: <4B4D0BD4.5050109@voidspace.org.uk>

On 12/01/2010 23:40, Michael Foord wrote:
> On 12/01/2010 23:28, "Martin v. L?wis" wrote:
>> [snip...]
>> """The binaries for AMD64 will also work on processors that implement
>> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
>> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
>> They will not work on Intel Itanium Processors (formerly IA-64)."""
>>
>
> To those not intimately aware of the history and present of 64 bit 
> architecture this sentence would be a great addition - thanks. I'm 
> adding it now as a footnote.
>

I can add it now to the main download page and existing release pages - 
but are there changes needed to any scripts to change pages created for 
*new* releases?

All the best,

Michael Foord

> All the best,
>
> Michael Foord
>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From python at mrabarnett.plus.com  Wed Jan 13 01:22:01 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 00:22:01 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <hiiv75$5md$1@ger.gmane.org>
References: <4B4CF354.2050603@mrabarnett.plus.com> <hiiv75$5md$1@ger.gmane.org>
Message-ID: <4B4D1229.70708@mrabarnett.plus.com>

Terry Reedy wrote:
> On 1/12/2010 5:10 PM, MRAB wrote:
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
> 
> Are you writing a new module with a new name? If so, do you expect it to 
> replace or augment re? (This is the same question as for optparse vs. 
> argparse, which I understand to not yet be decided.)
 >
It's a module called 'regex'. It can be used in place of 're' by using
"import regex as re", except for differences such as "\g<name>" being a
legal group reference in pattern strings.
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
> 
> 2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
> 2.7 stdlib is already out. A new engine should get some community 
> testing before going in the stdlib. Even 3.2 beta is not that far off 
> (8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?
> 
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
> 
> What does re do with analogous cases?
> 
The 're' module treats r"\g" as "g"; both 're' and 'regex' treat, say, 
r"\q" as "q". The closest analogue to what I'm asking about is that re
treats the ill-formed repeat r"x{1," as a literal, which sort of
suggests that r"\g" should be treated as "g", but r"\g<name>" is now a
group reference (re would treat that as "g<name>". Does that sound
reasonable?


From fuzzyman at voidspace.org.uk  Wed Jan 13 01:50:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 00:50:39 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D18DF.6070502@voidspace.org.uk>

On 12/01/2010 23:41, Christian Heimes wrote:
> Michael Foord wrote:
>    
>> I presume the email below is about the Windows binary. Does the AMD64
>> release work on intel 64bit and can we make the wording clearer on the
>> download page?
>>
>> The current description is " Windows AMD64 binary".
>>      
> The installer works on all AMD64 compatible Intel CPUs. *scnr*
>
> As you most likely know all modern Intel 64bit CPUs are based on AMD's
> x86-64 design. Only the Itanium family is based on the other Intel 64bit
> design: IA-64. The name AMD64 was chosen because most Linux and BSD
> distributions call it so. The name 'AMD64' has caused confusion in the
> past because users thought that the installer works only on AMD CPUs.
>
> How about:
>
>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
>
> instead of:
>
>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)
>    
Right - I've made that change for the Python 2.6, 2.7, 3.0 and 3.1 
download pages with a footnote. Prior to that we were offering ia64 
*and* amd64 releases so I've left those unchanged.

Thanks,

Michael

> ?
>
> Christia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From exarkun at twistedmatrix.com  Wed Jan 13 02:40:03 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Jan 2010 01:40:03 -0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100113014003.1898.1305834328.divmod.xquotient.219@localhost.localdomain>

On 12 Jan, 10:04 pm, martin at v.loewis.de wrote:
>>[...]
>>>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>>>2.x can do nothing:
>>[...]
>>>Inherently, 2.8 can't improve on that.
>>
>>I agree that there are limitations like the ones you've listed, but I
>>disagree with your conclusion.  Maybe you assume that it's just as 
>>hard
>>to move to 2.8 (using the py3k backports not available in say 2.5) as 
>>it
>>is to 3.x?
>
>Not at all, no. I'd rather expect that code that runs on 2.7 will run
>on 2.8 unmodified.
>>But a hypothetical 2.8 would also give people a way to move closer to
>>py3k without giving up on using all their 2.x-only dependencies.
>
>How so? If they use anything that is new in 2.8, they *will* need to
>drop support for anything before it, no???
>>I think it's much more likely that libraries like Twisted can support 
>>2.8
>>in the near future than 3.x.
>
>Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
>that help Twisted in moving to 3.2?

I'm not reading this thread carefully enough to make any arguments on 
either side, but I can contribute a fact.

Twisted very likely does not support 2.8 today.  I base this on the fact 
that Twisted does not support 2.7 today, and I expect 2.8 will be more 
like 2.7 than it will be like 2.3 - 2.6 (which Twisted does support).

When I say "support" here, I mean "all of the Twisted unit tests pass on 
it".

Jean-Paul


From tseaver at palladion.com  Wed Jan 13 03:57:46 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 12 Jan 2010 21:57:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <4B4D36AA.7020607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Karen Tracey wrote:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>>
> Well yay.  Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.  That's
> down from 104 failures and 40 errors with 2.7 alpha 1.
> 
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Just to add another "success" data point:  Zope2's trunk, as well as the
2.12 release, passes all tests (2535 on the trunk) and brings up the
appserver just fine under 2.7a2.

There is an obnoxious deprecation warning out of the distutils:

  DeprecationWarning: 'compiler' specifies the compiler type in
  build_ext. If you want to get the compiler object itself, use
  'compiler_obj'

which is likely a simple one-line fix, if I only knew what the hell it
was whining about. ;)  The warning is extra obnoxious because it doesn't
tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
argument).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktNNqQACgkQ+gerLs4ltQ7d5gCcCGKVjyOlxnrAln0UnRibS7kd
lNIAoIs1RlSGMtJWaY11BqptfDmQvR87
=mIOO
-----END PGP SIGNATURE-----



From python at mrabarnett.plus.com  Wed Jan 13 04:09:34 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 03:09:34 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <4B4D396E.4050500@mrabarnett.plus.com>

MRAB wrote:
> Hi all,
> 
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
> 
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
> 
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
> 
> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
> 
I've just noticed something odd about the re module: the sub() method
doesn't take 'pos' or 'endpos' arguments. search() does; match() does;
findall() does(); finditer() does; but sub() doesn't. Maybe there has
never been a demand for it. (Nor split(), for that matter.)


From brett at python.org  Wed Jan 13 04:58:11 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 19:58:11 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>

On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:

> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
>
>
If it is a separate module under a different name it can do the proper
thing. People will just need to be aware of the difference when they import
the module.


> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
>
>
That's totally up to you. There is practically no chance of it getting into
the 2.x under the stdlib at this point since 2.7b1 is coming up and this
module has not been out in the wild for a year (to my knowledge).  If you
want to support 2.x that's fine and I am sure users would appreciate it, but
it isn't necessary to get into the Python 3 stdlib.


> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
>
>
If you want to minimize the differences then it should probably match. As I
said, since it is a different name to import under it can deviate where
reasonable, just make sure to clearly document the deviations.

-Brett



> Thanks
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/f1f73d56/attachment-0006.htm>

From martin at v.loewis.de  Wed Jan 13 07:11:49 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 07:11:49 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D6425.3060307@v.loewis.de>

> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
> 
> instead of:
> 
>  * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)

-1. AMD doesn't want us to use the term x86-64 anymore, but wants us
to use AMD64 instead. I think we should comply - they invented the
architecture, so they have the right to give it a name. Neither
Microsoft nor Intel have such a right.

Regards,
Martin


From sridharr at activestate.com  Wed Jan 13 07:47:38 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 12 Jan 2010 22:47:38 -0800
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D6C8A.7070307@activestate.com>

On 1/12/2010 2:46 PM, Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?
>
> The current description is " Windows AMD64 binary".

FWIW, we simply use (64-bit, x64).

   Platform	          Download
   ==============================================
   Windows (x86)	          Windows Installer (MSI)
   Windows (64-bit, x64)	  Windows Installer (MSI)

https://www.activestate.com/activepython/downloads/

-srid


From solipsis at pitrou.net  Wed Jan 13 08:08:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 13 Jan 2010 07:08:55 +0000 (UTC)
Subject: [Python-Dev] Fwd: Download Page - AMD64
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <loom.20100113T080717-468@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)

+1. I don't care about trademarks or official names, we should call it whatever
is obvious for our users.
As for Itanium, it is practically dead, I think there is little chance of people
mixing it up with x86-64.

cheers

Antoine.




From michael at voidspace.org.uk  Wed Jan 13 10:32:00 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 09:32:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <4B4D9310.6060907@voidspace.org.uk>

On 13/01/2010 06:11, "Martin v. L?wis" wrote:
>> How about:
>>
>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
>>
>> instead of:
>>
>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>> not include source)
>>      
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.
>    

I think we should use whatever is most informative and least confusing 
to our users - we owe our allegiance to them and not to a processor vendor.

Michael


> Regards,
> Martin
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ncoghlan at gmail.com  Wed Jan 13 11:30:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:30:50 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF17A.1000503@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk>
Message-ID: <4B4DA0DA.7070007@gmail.com>

Michael Foord wrote:
> I agree with Martin that the *perception* is that to use Python 2.6 to
> help you port to Python 3 you have to be willing to drop support for
> earlier versions of Python.

Note that increased 3.x compatibility in the most recent 2.x release
will always help in two scenarios:

1. New projects that want to use 2.x only libraries but want to be ready
for the Py3k transition in their own code (e.g. new 2.7 features like
set literals, dict and set comprehensions and the view* dict methods can
be translated far more cleanly by 2to3 than the closest comparable 2.6
code).

2. Similarly new projects that use a 3.x trunk can be more readily
translated to a 2.7 version with 3to2, whereas some constructs may be
difficult to recreate in earlier Python versions. I would expect this to
be significantly less common then the first scenario though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:38:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:38:31 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <loom.20100113T080717-468@post.gmane.org>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<loom.20100113T080717-468@post.gmane.org>
Message-ID: <4B4DA2A7.2080408@gmail.com>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> How about:
>>
>>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
> 
> +1. I don't care about trademarks or official names, we should call it whatever
> is obvious for our users.

Until this discussion, I didn't even know the architecture had any name
other than x86-64.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:39:40 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:39:40 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4D36AA.7020607@palladion.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com>
Message-ID: <4B4DA2EC.3050908@gmail.com>

Tres Seaver wrote:
> There is an obnoxious deprecation warning out of the distutils:
> 
>   DeprecationWarning: 'compiler' specifies the compiler type in
>   build_ext. If you want to get the compiler object itself, use
>   'compiler_obj'
> 
> which is likely a simple one-line fix, if I only knew what the hell it
> was whining about. ;)  The warning is extra obnoxious because it doesn't
> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
> argument).

Could you kick a tracker issues in Tarek's direction for that one?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From vinay_sajip at yahoo.co.uk  Wed Jan 13 13:24:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 13 Jan 2010 12:24:09 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <loom.20100113T131816-515@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> If there is something missing from the stdlib discussion that you think should
be brought up at the summit let me know. And if there is something here you want
to discuss before the summit let me know and I can start a separate thread on it.

I'm sorry I won't be able to come to the language summit, but I would like if
possible to expedite a pronouncement on PEP 391 (configuring logging using
dictionaries). I believe I addressed all the comments made on the discussion
threads mentioned in the PEP and so I'm not sure what more I need to do to get a
pronouncement. I guess the stdlib slot gives an opportunity for people to air
their views and so I'd be grateful if you added it to the agenda.

Thanks & regards,

Vinay Sajip



From murman at gmail.com  Wed Jan 13 14:35:51 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 07:35:51 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>

On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.

I see and agree with the motivation behind your point, but it's just
as reasonable to mimic the name Microsoft uses: the release is for
Windows rather than the processor. On MSDN Microsoft uses
parentheticals x86, ia64 and x64; this would suggest a name like
Python 2.6.4 installer for Windows (x64). Unfortunately this usage
doesn't seem to be reflected in consumer-oriented product pages, so
I'm uncertain how clear it is for those downloading Python.

-- 
Michael Urman


From ncoghlan at gmail.com  Wed Jan 13 15:04:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 00:04:28 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
Message-ID: <4B4DD2EC.3030709@gmail.com>

Michael Urman wrote:
> On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
> 
> I see and agree with the motivation behind your point, but it's just
> as reasonable to mimic the name Microsoft uses: the release is for
> Windows rather than the processor. On MSDN Microsoft uses
> parentheticals x86, ia64 and x64; this would suggest a name like
> Python 2.6.4 installer for Windows (x64). Unfortunately this usage
> doesn't seem to be reflected in consumer-oriented product pages, so
> I'm uncertain how clear it is for those downloading Python.

As Windows doesn't run on non-x86 architectures, their naming is
generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Linux, on the other hand, can run on multiple 64 bit architectures, so
being more specific and using the official AMD64 nomenclature makes sense.

In this case, making it clear that the 64-bit Windows binaries work for
both Intel and AMD chips is more important than using only the official
architecture name.

Cheers,
Nick.

P.S. This discussion made me realise that out of habit I had dropped the
32-bit version of Python on my new 64-bit Windows box...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tseaver at palladion.com  Wed Jan 13 17:49:55 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 13 Jan 2010 11:49:55 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA2EC.3050908@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com> <4B4DA2EC.3050908@gmail.com>
Message-ID: <4B4DF9B3.4030403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> There is an obnoxious deprecation warning out of the distutils:
>>
>>   DeprecationWarning: 'compiler' specifies the compiler type in
>>   build_ext. If you want to get the compiler object itself, use
>>   'compiler_obj'
>>
>> which is likely a simple one-line fix, if I only knew what the hell it
>> was whining about. ;)  The warning is extra obnoxious because it doesn't
>> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
>> argument).
>
> Could you kick a tracker issues in Tarek's direction for that one?

Done:

  http://bugs.python.org/issue7694


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktN+bMACgkQ+gerLs4ltQ6s4gCdENhtVQWzoH449BCDz8SNx/4s
DEoAn19LMcMs3b6ctdNF8K2hYd6d+BSZ
=TWit
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Jan 13 18:24:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jan 2010 12:24:42 -0500
Subject: [Python-Dev] PYTHON3PATH
Message-ID: <20100113172442.4A7771FA679@kimball.webabinitio.net>

Please review issue 2375 [1], which is an enhancement request to add a
PYTHON3PATH environment variable.  Because we have elected to have both
a python and a python3 command, I think this is an issue worth thinking
about carefully to make sure we are serving the Python user community
and easing the transition to python3.  It could be that "use virtualenv"
is the best answer, but I feel we should think about it carefully to
make sure that is really true.

[1] http://bugs.python.org/issue2375

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From guido at python.org  Wed Jan 13 18:31:39 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:31:39 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>

Somehow the bug site doesn't load for me right now, but I'm -1 on
this. There are maybe a dozen PYTHON... variables -- we really
shouldn't try to add PYTHON3 variants for all of them.

Specifically for PYTHONPATH, I feel that its use is always a
short-time or localized hack, not something you set in your .profile
nd forget about it (forgetting about it inparticular often leads to
unnecessary debugging pain later). The better approaches are based on
site-packages, wrapper scripts, virtualenv, etc.

--Guido

On Wed, Jan 13, 2010 at 9:24 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable. ?Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3. ?It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375
>
> --
> R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From dirkjan at ochtman.nl  Wed Jan 13 18:38:26 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 13 Jan 2010 18:38:26 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <ea2499da1001130938v68827914yc41476c75b5f3c62@mail.gmail.com>

On Sat, Jan 9, 2010 at 18:29, Benjamin Peterson <benjamin at python.org> wrote:
> Please consider trying Python 2.7 with your code and reporting any bugs you may

I ran the Mercurial test suite on current trunk, and it worked
alright. There was a small issue with the fact that mimetypes got
fooled by the lack of ImportError from _winreg (which I solved by
blacklisting _winreg in demandimport), and one test fails likely
because of a change in MIME handling. Will look into that. So the
state of trunk looks rather solid from where I sit.

Cheers,

Dirkjan


From ralf at brainbot.com  Wed Jan 13 18:40:04 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:40:04 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net> (R. David
	Murray's message of "Wed, 13 Jan 2010 12:24:42 -0500")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <87r5ptdhu3.fsf@muni.brainbot.com>

"R. David Murray" <rdmurray at bitdance.com> writes:

> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable.  Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3.  It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375

The first thing I got while trying to run a python3 prompt few days ago,
was an error. python3 tried to read my $PYTHONSTARTUP file, which used
print statements. people will have to run both python 2 and python 3
code at the same time. Using different environment variables will make
this easier.

- Ralf


From ralf at brainbot.com  Wed Jan 13 18:55:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:55:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
	(Guido van Rossum's message of "Wed, 13 Jan 2010 09:31:39 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
Message-ID: <87k4vldh4c.fsf@muni.brainbot.com>

Guido van Rossum <guido at python.org> writes:

> Somehow the bug site doesn't load for me right now, but I'm -1 on
> this. There are maybe a dozen PYTHON... variables -- we really
> shouldn't try to add PYTHON3 variants for all of them.
>
> Specifically for PYTHONPATH, I feel that its use is always a
> short-time or localized hack, not something you set in your .profile
> nd forget about it (forgetting about it inparticular often leads to
> unnecessary debugging pain later). The better approaches are based on
> site-packages, wrapper scripts, virtualenv, etc.

then at least remove them completely from python3, so one can use them
for his python 2 installation. Otherwise it will stump on some innocent
python 3 program (and cause unnecessary debugging pain).



From steven.bethard at gmail.com  Wed Jan 13 18:57:29 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 13 Jan 2010 09:57:29 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:40 AM, Ralf Schmitt <ralf at brainbot.com> wrote:
> "R. David Murray" <rdmurray at bitdance.com> writes:
>
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable. ?Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3. ?It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>>
>> [1] http://bugs.python.org/issue2375
>
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

How complicated is your PYTHONSTARTUP file? My suspicion is that you
could easily write it to work for both Python 2.X and 3.X.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From ssteinerx at gmail.com  Wed Jan 13 18:57:42 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 13 Jan 2010 12:57:42 -0500
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>


On Jan 13, 2010, at 12:40 PM, Ralf Schmitt wrote:

> "R. David Murray" <rdmurray at bitdance.com> writes:
> 
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable.  Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3.  It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>> 
>> [1] http://bugs.python.org/issue2375
> 
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

Or, how about just removing the antiquated use of environment variables altogether from Python 3 and avoid the issue completely.  

Python 2 will continue to run as it has, and better solutions will emerge for Python 3 development.

Beats the heck out of making a whole duplicate set for PYTHON3BLAHBLAH, yuck!

S



From guido at python.org  Wed Jan 13 18:58:04 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:58:04 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
	<bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
Message-ID: <ca471dc21001130958k6edabaacof256b5e811c75ea6@mail.gmail.com>

Memories of days past... Python had several regular expression
implementations before, one of which was called "regex".

But I would rather not have a new module -- I would much rather have a
flag specifying the new (backwards incompatible) syntax/semantics. The
flag would have a long name (e.g. re.NEW_SYNTAX), a short name (e.g.
re.N) and an inline syntax, "(?n)...".

--Guido

On Tue, Jan 12, 2010 at 7:58 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
>>
>
> If it is a separate module under a different name it can do the proper
> thing. People will just need to be aware of the difference when they import
> the module.
>
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
>>
>
> That's totally up to you. There is practically no chance of it getting into
> the 2.x under the stdlib at this point since 2.7b1 is coming up and this
> module has not been out in the wild for a year (to my knowledge). ?If you
> want to support 2.x that's fine and I am sure users would appreciate it, but
> it isn't necessary to get into the Python 3 stdlib.
>
>>
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
>>
>
> If you want to minimize the differences then it should probably match. As I
> said, since it is a different name to import under it can deviate where
> reasonable, just make sure to clearly document the deviations.
> -Brett
>
>>
>> Thanks
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)


From ralf at brainbot.com  Wed Jan 13 19:03:08 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 19:03:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
	(Steven Bethard's message of "Wed, 13 Jan 2010 09:57:29 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
Message-ID: <87fx69dgrn.fsf@muni.brainbot.com>

Steven Bethard <steven.bethard at gmail.com> writes:

>
> How complicated is your PYTHONSTARTUP file? My suspicion is that you
> could easily write it to work for both Python 2.X and 3.X.

sure. that's exactly what I did. My point is that sharing those
environment variables will cause pain for some people.


From guido at python.org  Wed Jan 13 19:07:58 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:07:58 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
just removing the antiquated use of environment variables altogether
from Python 3 and avoid the issue completely.

-1. They have their use, but more in controlled situations. If you
have "global" env vars that you only want to use with Python 2.x,
write a wrapper for Python 3 that invokes it with -E.

-- 
--Guido van Rossum (python.org/~guido)


From scott+python-dev at scottdial.com  Wed Jan 13 19:14:21 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 13 Jan 2010 13:14:21 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E0D7D.3040806@scottdial.com>

On 1/13/2010 9:04 AM, Nick Coghlan wrote:
> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

That is not correct. There are IA-64 versions of Window Server.

>From [1]:
"Backward compatibility is a key point differentiating Itanium from x86
and x64 architectures."

So to echo what Michael said, the Microsoft nomenclature is "x64"
regardless of yours and Martin's objections to that name. Nobody who
uses Windows would be confused by "x64" since that is *the* Microsoft
naming scheme. And, anyone using IA64 will expect to see "IA64" and not
"x64", and furthermore anyone using IA64 is likely aware of what
processor they are using (being as there are only Windows Server
editions for IA64) and the difference between "IA64" and "x64".

I don't think an end-user facing page is a great place for pedantic and
wordy defenses of defying the Microsoft-blessed naming scheme.

> Linux, on the other hand, can run on multiple 64 bit architectures, so
> being more specific and using the official AMD64 nomenclature makes
> sense.

This has no relevance to the conversation since there are no Linux
binaries being distributed. The conversation on the expectations of
Windows end-users, who are the target of the download links.

[1] http://www.microsoft.com/servers/64bit/itanium/overview.mspx

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From guido at python.org  Wed Jan 13 19:22:33 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:22:33 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <loom.20100113T131816-515@post.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<loom.20100113T131816-515@post.gmane.org>
Message-ID: <ca471dc21001131022w429e4ceal899235bc711ecaca@mail.gmail.com>

On Wed, Jan 13, 2010 at 4:24 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> If there is something missing from the stdlib discussion that you think should
> be brought up at the summit let me know. And if there is something here you want
> to discuss before the summit let me know and I can start a separate thread on it.
>
> I'm sorry I won't be able to come to the language summit, but I would like if
> possible to expedite a pronouncement on PEP 391 (configuring logging using
> dictionaries). I believe I addressed all the comments made on the discussion
> threads mentioned in the PEP and so I'm not sure what more I need to do to get a
> pronouncement. I guess the stdlib slot gives an opportunity for people to air
> their views and so I'd be grateful if you added it to the agenda.

Is the PEP in the stage of consensus yet? If so it may not even be
necessary to discuss it at the summit (though I certainly won't stop
people if they want to).

-- 
--Guido van Rossum (python.org/~guido)


From phd at phd.pp.ru  Wed Jan 13 19:18:50 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 21:18:50 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <20100113181850.GA3837@phd.pp.ru>

On Wed, Jan 13, 2010 at 12:57:42PM -0500, ssteinerX at gmail.com wrote:
> Or, how about just removing the antiquated use of environment variables

   "antiquated"? You are kidding, aren't you?!

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From guido at python.org  Wed Jan 13 19:51:14 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:51:14 -0800
Subject: [Python-Dev] PyCon Keynote
Message-ID: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>

Please mail me topics you'd like to hear me talk about in my keynote
at PyCon this year.

-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Wed Jan 13 20:13:58 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:13:58 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D9310.6060907@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de> <4B4D9310.6060907@voidspace.org.uk>
Message-ID: <4B4E1B76.4010309@v.loewis.de>

>>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>> X86-64 binary -- does not include source)
>>>
>>> instead of:
>>>
>>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>> not include source)
>>>      
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
>>    
> 
> I think we should use whatever is most informative and least confusing
> to our users - we owe our allegiance to them and not to a processor vendor.

And why do you think this is x86-64?

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:33:24 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:33:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA0DA.7070007@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk> <4B4DA0DA.7070007@gmail.com>
Message-ID: <4B4E2004.8050905@v.loewis.de>

> Note that increased 3.x compatibility in the most recent 2.x release
> will always help in two scenarios:
> 
> 1. New projects that want to use 2.x only libraries but want to be ready
> for the Py3k transition in their own code (e.g. new 2.7 features like
> set literals, dict and set comprehensions and the view* dict methods can
> be translated far more cleanly by 2to3 than the closest comparable 2.6
> code).

Of these, I can only see this being the case of the view* methods.

Why would set literals allow for code that more cleanly translates
to 3.x? Suppose you write

 f = set((1,2,3))

in 2.6. That code will work *exactly* the same way in 3.1, no
translation needed at all. Likewise for dict and set comprehensions:
the code is as cleanly translated in the 2.7 form as it is in any
prior form (ie. no modifications).

As for view* methods: there is one important exception where
the 2.6 equivalent is as cleanly translated as a view method:

for key in D:
  action

This is a more modern equivalent for iterating over D.keys(),
so if your code still does the latter, just change it to the
former (requires Python 2.2). I'd claim that this is the dominant
case of traversing a dictionary (probably followed by iterating
over .items()). So while it is true that only view* can be
transformed correctly, I'd claim that this is an infrequent
issue.

> 2. Similarly new projects that use a 3.x trunk can be more readily
> translated to a 2.7 version with 3to2, whereas some constructs may be
> difficult to recreate in earlier Python versions.

That may be true, but is besides the point here: the issue was whether
a 2.8 release might help in migrating to 3.x. People who follow this
approach already have 3.x code. Whether they would rather port to 2.8
only, and get that work with little effort, or whether they would port
to 2.5 and later, and put in more work, I don't know - I guess we would
have to ask these people. I would expect that they prefer if 2.x
died rather sooner than later, because they can then stop supporting
it.

Regards,
Martin



From tjreedy at udel.edu  Wed Jan 13 20:36:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 Jan 2010 14:36:03 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <hil7av$52r$1@ger.gmane.org>

On 1/13/2010 2:13 PM, "Martin v. L?wis" wrote:

>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>
> And why do you think this is x86-64?

I do not believe I have personally seen, or at least noticed, that 
specific term before. Something like

"Release for 64-bit Windows (Win NT, 2000, XP, Vista, 7 <or whatever 
correct list is>) on AMD64-type processors from AMD and Intel"

should be clear enough.




From martin at v.loewis.de  Wed Jan 13 20:40:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 20:40:30 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E21AE.40602@v.loewis.de>

> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Windows actually does - it runs on IA-64 (which is a non-x86 architecture).

However, end users are unlikely to use such hardware, so distinguishing
between 32-bit and 64-bit is typically good enough.

> In this case, making it clear that the 64-bit Windows binaries work for
> both Intel and AMD chips is more important than using only the official
> architecture name.

Well, we did have Itanium binaries at one point, and the "64-bit Windows
binaries" you are referring to most definitely don't run on Itanium,
even though that's an Intel chip.

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:45:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:45:40 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E0D7D.3040806@scottdial.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>	<4B4DD2EC.3030709@gmail.com>
	<4B4E0D7D.3040806@scottdial.com>
Message-ID: <4B4E22E4.4040504@v.loewis.de>

> So to echo what Michael said, the Microsoft nomenclature is "x64"
> regardless of yours and Martin's objections to that name. Nobody who
> uses Windows would be confused by "x64" since that is *the* Microsoft
> naming scheme.

That's actually not entirely true. There are several places in the
APIs where Microsoft either allows or requires to call the architecture
AMD64 (e.g. architecture indication in MSI files).

Regards,
Martin


From regebro at gmail.com  Wed Jan 13 20:50:59 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 20:50:59 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>

On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

What do you need to do in the PYTHONSTARTUP file?
Ten years of Python programming, and I didn't even know it existed. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From phd at phd.pp.ru  Wed Jan 13 21:08:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 23:08:40 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <20100113200840.GC14858@phd.pp.ru>

On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From murman at gmail.com  Wed Jan 13 21:12:11 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 14:12:11 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E22E4.4040504@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com> <4B4E0D7D.3040806@scottdial.com>
	<4B4E22E4.4040504@v.loewis.de>
Message-ID: <dcbbbb411001131212q3ef907c8i67e8c03c96c31e2f@mail.gmail.com>

On Wed, Jan 13, 2010 at 13:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So to echo what Michael said, the Microsoft nomenclature is "x64"
>> regardless of yours and Martin's objections to that name. Nobody who
>> uses Windows would be confused by "x64" since that is *the* Microsoft
>> naming scheme.
>
> That's actually not entirely true. There are several places in the
> APIs where Microsoft either allows or requires to call the architecture
> AMD64 (e.g. architecture indication in MSI files).

I should have clarified I was talking about the names shown on MSDN
subscriptions for downloading installation media of Windows 7 and
Windows Vista. It's pretty clear in that context Microsoft uses x64 to
describe this platform of Windows. But again, it's far from clear that
this is a term they use for non-developers.

-- 
Michael Urman


From regebro at gmail.com  Wed Jan 13 21:27:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 21:27:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113200840.GC14858@phd.pp.ru>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<20100113200840.GC14858@phd.pp.ru>
Message-ID: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>

On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>> What do you need to do in the PYTHONSTARTUP file?
>> Ten years of Python programming, and I didn't even know it existed. :-)
>
> ? See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Cheese and fries! :-)

Anyway, I don't see how this could pose any problems, because any user
advanced enough to do these things will be advanced enough to
understand what the problem is and fix it so it works under Python 3
too.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ralf at brainbot.com  Wed Jan 13 21:52:34 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 20:52:34 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	(Lennart Regebro's message of "Wed, 13 Jan 2010 20:50:59 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <874ompg225.fsf@brainbot.com>

Lennart Regebro <regebro at gmail.com> writes:

> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
>
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

hehe. tab completion:

# -*- mode: python -*- 
# Last changed: 2009-12-23 22:25:15 by ralf

import sys
import os

def initreadline():
    
    try:
        import readline
    except ImportError:
        sys.stdout.write("Module readline not available.\n")
        return
    
    import rlcompleter
    readline.parse_and_bind("tab: complete")
    
    # Use tab for completions
    readline.parse_and_bind('tab: complete')
    # This forces readline to automatically print the above list when tab
    # completion is set to 'complete'.
    readline.parse_and_bind('set show-all-if-ambiguous on')
    # Bindings for incremental searches in the history. These searches
    # use the string typed so far on the command line and search
    # anything in the previous input history containing them.
    readline.parse_and_bind('"\C-r": reverse-search-history')
    readline.parse_and_bind('"\C-s": forward-search-history')

    history = os.path.expanduser("~/.pyhistory") 
    if os.path.exists(history):
        readline.read_history_file(history)
        
    import atexit
    atexit.register(lambda: readline.write_history_file(history))
    
initreadline()
del initreadline


From martin at v.loewis.de  Wed Jan 13 22:05:03 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 22:05:03 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <4B4E357F.4050107@v.loewis.de>

Lennart Regebro wrote:
> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
> 
> What do you need to do in the PYTHONSTARTUP file?

I personally set up readline: set tab completion, and load the history
of commands from the previous session.

Of course, I don't know what Ralf uses it for.

Regards,
Martin


From ncoghlan at gmail.com  Wed Jan 13 22:43:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 07:43:41 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
Message-ID: <4B4E3E8D.2010407@gmail.com>

Guido van Rossum wrote:
> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
> just removing the antiquated use of environment variables altogether
> from Python 3 and avoid the issue completely.
> 
> -1. They have their use, but more in controlled situations. If you
> have "global" env vars that you only want to use with Python 2.x,
> write a wrapper for Python 3 that invokes it with -E.

Perhaps a case can be made for Python 3 to assume -E by default, with a
-e option to enable reading of the environment variables?

That way naive users could run Python3 without tripping over existing
Python2 environment variables, while other tools could readily set up a
different environment before launching Python3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From guido at python.org  Wed Jan 13 23:45:57 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 14:45:57 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com> 
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com> 
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <ca471dc21001131445g639c6867ude83f03a77eb72b9@mail.gmail.com>

On Wed, Jan 13, 2010 at 1:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
>
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
>
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users using environment variables? That's a recipe for disaster
in any version! :-)

-- 
--Guido van Rossum (python.org/~guido)


From jared.grubb at gmail.com  Wed Jan 13 23:56:37 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Wed, 13 Jan 2010 14:56:37 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <13F6C43A-D62A-4A9F-AF7A-9BDAC94F7D72@gmail.com>

On 13 Jan 2010, at 13:43, Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>> 
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

I raised a question about these PYTHON3* variables once before in a discussion about shebang lines:
http://www.mailinglistarchive.com/python-dev at python.org/msg29500.html

I'm not advocating them, but just wanted to make sure to bring up the shebang use case.

Jared

From skip at pobox.com  Wed Jan 13 23:24:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 16:24:13 -0600
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <19278.18445.428716.321365@montanaro.dyndns.org>


    Lennart> What do you need to do in the PYTHONSTARTUP file?

Just reading off stuff from my own personal startup file...  I use it for
stuff I want available during interactive sessions:

    1. Enable true division.

    2. Conditionally define "help" from back in the days when there was no
       help builtin function.

    3. Auto-save session (readline) history so I can easily recall commands
       across sessions.

    4. Add other fake builtins ("like") or override behavior of some (like
       "dir") making them handier for interactive use.

    5. autoload modules/symbols (pokes around in common modules from
       sys.excepthook function).

Oh, and I've had no particular trouble keeping it working in Python 1, 2 or
3.

Skip




From fuzzyman at voidspace.org.uk  Thu Jan 14 01:20:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 00:20:21 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <4B4E6345.5070202@voidspace.org.uk>

On 13/01/2010 19:13, "Martin v. L?wis" wrote:
>>>>    * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>>> X86-64 binary -- does not include source)
>>>>
>>>> instead of:
>>>>
>>>>    * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>>> not include source)
>>>>
>>>>          
>>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>>> to use AMD64 instead. I think we should comply - they invented the
>>> architecture, so they have the right to give it a name. Neither
>>> Microsoft nor Intel have such a right.
>>>
>>>        
>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>>      
> And why do you think this is x86-64?
>    

Well anecdotal everyone I have *every* talked to about 64bit processors 
has referred to having a 64bit processor (x86 is a given) and not an 
AMD64 architecture processor.

Linus Torvalds addressed this specific issue for Linux and came down on 
the side of "x86-64": http://kerneltrap.org/node/2466
Look up AMD64 on Wikipedia and it redirects you to the X86-64 page.
Information website setup by AMD and partners about the AMD64 
architecture: http://www.x86-64.org/about.html
In the AMD website they refer to "x86-64 Assembly": 
http://www.x86-64.org/documentation/assembly.html

Microsoft seem to draw a distinction between x64 (which would also be 
acceptable) and Itanium based systems. Very rarely do they refer to AMD64:

* http://www.microsoft.com/servers/64bit/compare.mspx
* http://www.microsoft.com/servers/64bit/x64/overview.mspx
* http://www.microsoft.com/servers/64bit/overview.mspx

Using a vendor specific name automatically begs the question as to 
whether the installer works on processors from other vendors, as we saw 
in the specific enquiry from the user that triggered this debate.

Referring to the AMD 64 build as x86-64, with a footnote explaining 
which architectures this specifically means is unlikely to confuse 
people. It is *definitely* better than just saying AMD64.

All the best,

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From skip at pobox.com  Thu Jan 14 02:50:55 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 19:50:55 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
Message-ID: <19278.30847.649228.115514@montanaro.dyndns.org>


Just out of curiosity, is the static build stuff (use the old Modules/Setup
file to build modules) exercised at all any more?

Skip



From benjamin at python.org  Thu Jan 14 03:22:03 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 13 Jan 2010 20:22:03 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
In-Reply-To: <19278.30847.649228.115514@montanaro.dyndns.org>
References: <19278.30847.649228.115514@montanaro.dyndns.org>
Message-ID: <1afaf6161001131822r151bd202x35653ba44ee0235d@mail.gmail.com>

2010/1/13  <skip at pobox.com>:
>
> Just out of curiosity, is the static build stuff (use the old Modules/Setup
> file to build modules) exercised at all any more?

Exercised as in used in the build process? Yes.


-- 
Regards,
Benjamin


From vinay_sajip at yahoo.co.uk  Thu Jan 14 10:23:47 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 09:23:47 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
Message-ID: <954450.26617.qm@web25804.mail.ukl.yahoo.com>

In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:

  http://www.python.org/dev/peps/pep-0391/

In November 2009 I posted to this list that the PEP was ready for review.

I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.

So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.

Thanks & regards,

Vinay Sajip



      



From ziade.tarek at gmail.com  Thu Jan 14 10:30:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 Jan 2010 10:30:15 +0100
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <94bdd2611001140130u6154e893jfd37db32a25be66a@mail.gmail.com>

On Thu, Jan 14, 2010 at 10:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[..]
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would
> be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check
> in the changes unless there are objections to doing so. All those who think that logging is currently
> hard to configure will benefit from these changes, so here's the opportunity to pipe up and
> improve things.


FWIW, I am +1. Thanks for your work

Tarek


From hodgestar+pythondev at gmail.com  Thu Jan 14 10:39:41 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 14 Jan 2010 11:39:41 +0200
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <fb73205e1001140139n4b920871t6bfc6b91c469264f@mail.gmail.com>

On Thu, Jan 14, 2010 at 11:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I think the dict configuration scheme will be a great addition to the
logging module. :)

Schiavo
Simon


From p.f.moore at gmail.com  Thu Jan 14 12:22:09 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 Jan 2010 11:22:09 +0000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>

2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I've no immediate need for the feature, but it would be good to have
something like this, so I'm +1.
Paul.


From ncoghlan at gmail.com  Thu Jan 14 12:46:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:46:19 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
Message-ID: <4B4F040B.8010607@gmail.com>

Paul Moore wrote:
> 2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> So, can you please indicate your vote for or against incorporating PEP 391 into Python?
> 
> I've no immediate need for the feature, but it would be good to have
> something like this, so I'm +1.

I'm in the same boat as Paul, but PEP 291 provides a nice forward
compatible configuration scheme that will work with any application
configuration approach that can produce an appropriate Python dictionary.

So +1 from me too - I think the PEP has now taken this as far as
theorising can go, and the only way to learn anything further is to put
it into practice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Jan 14 12:57:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:57:55 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4B4F06C3.7030806@gmail.com>

Vinay Sajip wrote:
> In October 2009 I created PEP 391 to propose a new method of
> configuring logging using dictionaries:
> 
> http://www.python.org/dev/peps/pep-0391/

Although one minor comment: you can probably remove the note about the
"ext://" and "cfg://" prefixes being provisional at this stage. Those
names look fine to me, so I don't see any point inviting a late-breaking
bikeshed discussion about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Thu Jan 14 14:53:29 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 08:53:29 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>

On Thu, Jan 14, 2010 at 4:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:
>
> ?http://www.python.org/dev/peps/pep-0391/
>
> In November 2009 I posted to this list that the PEP was ready for review.
>
> I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.
>
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.
>
> Thanks & regards,
>
> Vinay Sajip

I'm generally +1 - but given I know that Django 1.2 is slated to
implement something somewhat similar, I'm interested to hear how this
proposal meshes with their plan(s).

jesse


From vinay_sajip at yahoo.co.uk  Thu Jan 14 15:08:24 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 14:08:24 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
Message-ID: <92210.91656.qm@web25806.mail.ukl.yahoo.com>

> From: Jesse Noller <jnoller at gmail.com>

> I'm generally +1 - but given I know that Django 1.2 is slated to
> implement something somewhat similar, I'm interested to hear how this
> proposal meshes with their plan(s)..

Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:

http://groups.google.com/group/django-developers/msg/4ef81a2160257221

They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).

Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

Regards,

Vinay Sajip


      



From ssteinerx at gmail.com  Thu Jan 14 15:54:27 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 14 Jan 2010 09:54:27 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <62E362ED-5DD7-4D42-B5E0-B263A137FD5C@gmail.com>


On Jan 14, 2010, at 9:08 AM, Vinay Sajip wrote:

>> From: Jesse Noller <jnoller at gmail.com>
> 
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
> 
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
> 
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
> 
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
> 
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

That puts a huge +1 on there for me.  

If we can get this approved and have Django's logging improved *and* avoid a whole pile of duplicate effort in Django that's a huge win.

S



From jnoller at gmail.com  Thu Jan 14 15:56:18 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 09:56:18 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140656w68c7290agccfe7282cf96f2fb@mail.gmail.com>

On Thu, Jan 14, 2010 at 9:08 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> From: Jesse Noller <jnoller at gmail.com>
>
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
>
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
>
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
>
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
>
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.
>
> Regards,
>
> Vinay Sajip
>

Cool, +1 then :)


From mal at egenix.com  Thu Jan 14 20:19:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 20:19:12 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <4B4F6E30.50400@egenix.com>

Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users won't have PYTHONPATH or any other Python environment
variables setup.

Really, if you know that you are going to run Python 3 instead of
Python 2 or vice-versa it's easy enough to run

. py3env.sh
or
. py2env.sh

in order to setup your shell environment, much like you typically
do when using virtual Python installations or work on different
projects that require different setups.

If you just want to separate Python 2 and 3 files, use the
user site-packages dir which includes the Python version.

More experienced users could also write their own environment
switching sitecustomize.py or usercustomize.py.

And then there's the hackery option for experts that love
dirty tricks: add a .pth file which includes an "import mysetup"
line to fix-up you path and other settings.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Jan 14 22:02:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 15 Jan 2010 07:02:28 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <4B4F8664.4080107@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Guido van Rossum wrote:
>>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>>> just removing the antiquated use of environment variables altogether
>>> from Python 3 and avoid the issue completely.
>>>
>>> -1. They have their use, but more in controlled situations. If you
>>> have "global" env vars that you only want to use with Python 2.x,
>>> write a wrapper for Python 3 that invokes it with -E.
>> Perhaps a case can be made for Python 3 to assume -E by default, with a
>> -e option to enable reading of the environment variables?
>>
>> That way naive users could run Python3 without tripping over existing
>> Python2 environment variables, while other tools could readily set up a
>> different environment before launching Python3.
> 
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.

I was actually thinking of the situation where the OS had a preinstalled
Python 2 with a default PYTHONPATH setting and the user was playing with
Python 3.

However, I agree that that is a fairly unlikely scenario (since
preinstalled Pythons tend not to rely on the environment variables and a
user sophisticated enough to be playing with a new Python interpreter
should be able to cope with a few environment variable conflicts anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at voidspace.org.uk  Thu Jan 14 22:09:30 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 21:09:30 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F8664.4080107@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <4B4F8664.4080107@gmail.com>
Message-ID: <4B4F880A.5010809@voidspace.org.uk>

On 14/01/2010 21:02, Nick Coghlan wrote:
> However, I agree that that is a fairly unlikely scenario (since
> preinstalled Pythons tend not to rely on the e
Well, on the other hand I think that during the next few years it will 
be increasingly common for developers (and possibly users) to have 
Python 2 and Python 3 installed side-by-side.

Many libraries and applications may never make the jump to Python 3 and 
Python users may be using 'legacy' Python 2 code for many years to come. 
It will also become increasingly common for developers to be using 
Python 3 *primarily* and for Python 3 only libraries and applications to 
emerge.

Whilst there are workarounds we *are* in a situation that Python 2 and 
Python 3 share environment variables for the location of libraries and 
executing code on startup, whilst at the same time they are largely 
incompatible and need separate library paths and startup code.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ralf at brainbot.com  Thu Jan 14 22:25:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 14 Jan 2010 22:25:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com> (M.'s message of "Thu, 14 Jan 2010
	20:19:12 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <871vhswf90.fsf@brainbot.com>

"M.-A. Lemburg" <mal at egenix.com> writes:

>
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.
>
> Really, if you know that you are going to run Python 3 instead of
> Python 2 or vice-versa it's easy enough to run

You don't even know that you're going to run python. I have 40 python
scripts in my /usr/bin directory. 

>
> . py3env.sh
> or
> . py2env.sh
>
> in order to setup your shell environment, much like you typically
> do when using virtual Python installations or work on different
> projects that require different setups.

No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.

>
> If you just want to separate Python 2 and 3 files, use the
> user site-packages dir which includes the Python version.

Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
install those programs in a user's home directory. There are still
people running python <2.6.


From mal at egenix.com  Thu Jan 14 22:51:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 22:51:07 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <871vhswf90.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <871vhswf90.fsf@brainbot.com>
Message-ID: <4B4F91CB.2060106@egenix.com>

Ralf Schmitt wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>
>> Naive users won't have PYTHONPATH or any other Python environment
>> variables setup.
>>
>> Really, if you know that you are going to run Python 3 instead of
>> Python 2 or vice-versa it's easy enough to run
> 
> You don't even know that you're going to run python. I have 40 python
> scripts in my /usr/bin directory. 
> 
>>
>> . py3env.sh
>> or
>> . py2env.sh
>>
>> in order to setup your shell environment, much like you typically
>> do when using virtual Python installations or work on different
>> projects that require different setups.
> 
> No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.
> 
>>
>> If you just want to separate Python 2 and 3 files, use the
>> user site-packages dir which includes the Python version.
> 
> Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
> install those programs in a user's home directory. There are still
> people running python <2.6.

Note that you only need to have a different PYTHONPATH
setups for Python 2.x/3.x if you plan to run code that:

 * is not installed in site-packages (most OS shipped code
   will be found in the resp. system site-packages/ dir)

 * is not installed in a user site-packages directory
   (user installed code will typically go there (*))

 * uses modules/packages that come in two different versions
   (one for Python 2.x and one for 3.x) and use the same name
   for both versions

 * needs to work in both Python 2.x and 3.x


(*) The method for installing code in user site-packages dir is
running:

    python setup.py install --user

instead of the standard

    python setup.py install

which install to the system-wide site-packages.

BTW: I guess the bzr and mercurial wikis will need to be updated
accordingly - at least for users of Python >=2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From martin at v.loewis.de  Thu Jan 14 22:55:04 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Jan 2010 22:55:04 +0100
Subject: [Python-Dev] [ANN] Python 2.5.5 Release Candidate 1.
Message-ID: <4B4F92B8.30806@v.loewis.de>

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.5.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.6 (which is
2.6.4 at this point).

This releases fixes issues with the logging and tarfile modules, and
with thread-local variables. See the detailed release notes at the
website (also available as Misc/NEWS in the source distribution) for
details of bugs fixed.

For more information on Python 2.5.5, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.5.5

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

    http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
martin at v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)


From status at bugs.python.org  Fri Jan 15 18:07:26 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jan 2010 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100115170726.9963A7865F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/08/10 - 01/15/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2536 open (+32) / 16993 closed (+16) / 19529 total (+48)

Open issues with patches:  1024

Average duration of open issues: 710 days.
Median duration of open issues: 469 days.

Open Issues Breakdown
   open  2501 (+32)
pending    34 ( +0)

Issues Created Or Reopened (53)
_______________________________

PYTHON3PATH environment variable to supersede PYTHONPATH for mul 01/13/10
CLOSED http://bugs.python.org/issue2375    reopened r.david.murray                
                                                                               

Mark the compiler package as deprecated                          01/13/10
       http://bugs.python.org/issue6837    reopened ezio.melotti                  
                                                                               

test_distutils failure                                           01/15/10
CLOSED http://bugs.python.org/issue6961    reopened flox                          
       buildbot                                                                

test_urllib: unsetting missing 'env' variable                    01/08/10
CLOSED http://bugs.python.org/issue7026    reopened flox                          
                                                                               

Two float('nan') are not equal                                   01/08/10
CLOSED http://bugs.python.org/issue7660    created  jmfauth                       
                                                                               

compiling ctypes fails with non-ascii path                       01/08/10
       http://bugs.python.org/issue7661    created  pitrou                        
       patch                                                                   

time.utcoffset()                                                 01/09/10
       http://bugs.python.org/issue7662    created  techtonik                     
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points  01/10/10
CLOSED http://bugs.python.org/issue7663    created  exarkun                       
                                                                               

python logger does not handle IOError Exception                  01/10/10
CLOSED http://bugs.python.org/issue7664    created  yateenjoshi                   
                                                                               

test_urllib2 and test_ntpath fail if path contains "\"           01/10/10
       http://bugs.python.org/issue7665    created  flox                          
                                                                               

test_lib2to3 fails if path contains space                        01/10/10
CLOSED http://bugs.python.org/issue7666    created  flox                          
                                                                               

test_doctest fails with non-ascii path                           01/10/10
       http://bugs.python.org/issue7667    created  flox                          
       buildbot                                                                

test_httpservers fails with non-ascii path                       01/10/10
       http://bugs.python.org/issue7668    created  flox                          
       buildbot                                                                

test_unicode_file fails with non-ascii path                      01/10/10
CLOSED http://bugs.python.org/issue7669    created  flox                          
                                                                               

_sqlite3: Block *all* operations on a closed Connection object   01/10/10
       http://bugs.python.org/issue7670    created  haypo                         
       patch                                                                   

test_popen fails if path contains special char like ";"          01/11/10
       http://bugs.python.org/issue7671    reopened flox                          
       patch                                                                   

_ssl module causes segfault                                      01/10/10
       http://bugs.python.org/issue7672    created  ssoria                        
       patch                                                                   

audioop: check that length is a multiple of the size             01/11/10
       http://bugs.python.org/issue7673    created  haypo                         
       patch                                                                   

select.select() corner cases: duplicate fds, out-of-range fds    01/11/10
       http://bugs.python.org/issue7674    created  cdleary                       
                                                                               

help() doesn't accept unicode literals in built in docstrings    01/11/10
       http://bugs.python.org/issue7675    created  psd                           
       patch                                                                   

IDLE shell shouldn't use TABs                                    01/11/10
       http://bugs.python.org/issue7676    created  cben                          
                                                                               

distutils, better error message for setup.py upload -sign withou 01/11/10
       http://bugs.python.org/issue7677    created  illume                        
                                                                               

subprocess.Popen pipeline example code in the documentation is l 01/11/10
       http://bugs.python.org/issue7678    created  steven.k.wong                 
                                                                               

Warning building 2.7 on OS X 10.6 libintl.h "Present But Cannot  01/12/10
       http://bugs.python.org/issue7679    created  ssteinerX                     
                                                                               

pythonw crash while attempting to start() a thread object        01/12/10
       http://bugs.python.org/issue7680    created  dontbugme                     
                                                                               

Incorrect division in [wave.py]                                  01/12/10
CLOSED http://bugs.python.org/issue7681    created  alfps                         
       patch, needs review                                                     

Optimisation of if with constant expression                      01/12/10
       http://bugs.python.org/issue7682    created  sdefresne                     
       patch                                                                   

Wrong link in HTML documentation                                 01/12/10
CLOSED http://bugs.python.org/issue7683    created  francescor                    
                                                                               

decimal.py: infinity coefficients in tuples                      01/12/10
       http://bugs.python.org/issue7684    created  skrah                         
                                                                               

minor typo in re docs                                            01/12/10
CLOSED http://bugs.python.org/issue7685    created  mikejs                        
       patch                                                                   

redundant open modes 'rbb', 'wbb', 'abb' no longer work on Windo 01/13/10
       http://bugs.python.org/issue7686    created  ivank                         
                                                                               

Bluetooth support untested                                       01/13/10
       http://bugs.python.org/issue7687    created  pitrou                        
                                                                               

TypeError: __name__ must be set to a string object               01/13/10
       http://bugs.python.org/issue7688    created  frankmillman                  
                                                                               

Pickling of classes with a metaclass and copy_reg                01/13/10
       http://bugs.python.org/issue7689    created  craigcitro                    
       patch                                                                   

Wrong PEP number in importlib module manual page                 01/13/10
CLOSED http://bugs.python.org/issue7690    created  francescor                    
                                                                               

test_bsddb3 files are not always removed when test fails         01/13/10
CLOSED http://bugs.python.org/issue7691    created  flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero              01/13/10
CLOSED http://bugs.python.org/issue7692    created  Bugger                        
                                                                               

tarfile.extractall can't have unicode extraction path            01/13/10
       http://bugs.python.org/issue7693    created  pbienst                       
                                                                               

DeprecationWarning from distuils.commands.build_ext needs stackl 01/13/10
       http://bugs.python.org/issue7694    created  tseaver                       
       patch                                                                   

missing termios constants                                        01/13/10
       http://bugs.python.org/issue7695    created  conorh                        
                                                                               

Improve Memoryview/Buffer documentation                          01/13/10
       http://bugs.python.org/issue7696    created  flox                          
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes  01/13/10
CLOSED http://bugs.python.org/issue7697    created  flox                          
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame  01/14/10
CLOSED http://bugs.python.org/issue7698    created  exarkun                       
       patch                                                                   

strptime, strftime documentation                                 01/14/10
       http://bugs.python.org/issue7699    created  mikejs                        
       patch                                                                   

"-3" flag does not work anymore                                  01/14/10
CLOSED http://bugs.python.org/issue7700    created  flox                          
       patch                                                                   

fix output string length for binascii.b2a_uu()                   01/14/10
CLOSED http://bugs.python.org/issue7701    created  haypo                         
       patch                                                                   

Wrong order of parameters of _get_socket in SMTP class in smtpli 01/14/10
       http://bugs.python.org/issue7702    created  alito                         
                                                                               

Replace buffer()-->memoryview() in Lib/ctypes/test/              01/14/10
       http://bugs.python.org/issue7703    created  flox                          
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE         01/14/10
CLOSED http://bugs.python.org/issue7704    created  DhaReaL                       
                                                                               

libpython2.6.so is not linked correctly on FreeBSD when threads  01/15/10
       http://bugs.python.org/issue7705    created  aballier                      
       patch, needs review                                                     

Missing #include guards                                          01/15/10
       http://bugs.python.org/issue7706    created  akrpic77                      
       patch                                                                   

multiprocess.Queue operations during import can lead to deadlock 01/15/10
       http://bugs.python.org/issue7707    created  kripken                       
                                                                               

Conversion of longs to bytes and vice-versa.                     01/11/10
       http://bugs.python.org/issue1023290 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (67)
______________________

segfault in curses when calling redrawwin() before refresh()      825 days
       http://bugs.python.org/issue1266    mark.dickinson                
                                                                               

"RuntimeError: dictionary changed size during iteration" in Tkin  751 days
       http://bugs.python.org/issue1658    flox                          
       patch                                                                   

Exception exceptions.AttributeError '_shutdown' in <module 'thre  718 days
       http://bugs.python.org/issue1947    nadiasvertex                  
                                                                               

Backport dictviews to 2.7                                         713 days
       http://bugs.python.org/issue1967    alexandre.vassalotti          
       patch, needs review                                                     

Backport set and dict comprehensions                              665 days
       http://bugs.python.org/issue2333    alexandre.vassalotti          
       patch, 26backport                                                       

Backport set literals                                             663 days
       http://bugs.python.org/issue2335    alexandre.vassalotti          
       patch, 26backport                                                       

PYTHON3PATH environment variable to supersede PYTHONPATH for mul    1 days
       http://bugs.python.org/issue2375    lemburg                       
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     625 days
       http://bugs.python.org/issue2698    cmcqueen1975                  
                                                                               

arg 2 of PyErr_SetFromErrnoWithFilename should be const           619 days
       http://bugs.python.org/issue2758    brian.curtin                  
                                                                               

Trunk build issues in DragonFly BSD                               613 days
       http://bugs.python.org/issue2797    brian.curtin                  
       patch, needs review                                                     

Gzip cannot handle zero-padded output + patch                     610 days
       http://bugs.python.org/issue2846    pitrou                        
       patch, needs review                                                     

block operation on closed socket/pipe for multiprocessing         552 days
       http://bugs.python.org/issue3311    haypo                         
       patch, needs review                                                     

Add gamma function, error functions and other C99 math.h functio  543 days
       http://bugs.python.org/issue3366    mark.dickinson                
       patch, needs review                                                     

Macros for PyLong: sign, number of digits, fits in an int         425 days
       http://bugs.python.org/issue4294    mark.dickinson                
       patch                                                                   

reject unicode in zlib                                            379 days
       http://bugs.python.org/issue4757    haypo                         
       patch                                                                   

Distutils inappropriately reuses .o files between extension modu  320 days
       http://bugs.python.org/issue5372    tarek                         
       patch, patch, needs review                                              

Problem with email.MIME* library, using wrong new line            298 days
       http://bugs.python.org/issue5525    r.david.murray                
                                                                               

os.path.normpath doesn't preserve unicode                         263 days
       http://bugs.python.org/issue5827    ezio.melotti                  
       patch                                                                   

smtplib exception smtp.connect TypeError encode_plain             173 days
       http://bugs.python.org/issue6523    r.david.murray                
                                                                               

email.parser clips trailing \n of multipart/mixed part if part e  152 days
       http://bugs.python.org/issue6681    r.david.murray                
       patch                                                                   

Optimize PyBytes_FromObject.                                      151 days
       http://bugs.python.org/issue6688    alexandre.vassalotti          
       patch                                                                   

in xmlrpclib.py: NameError: global name 'HTTPSConnection' is not  143 days
       http://bugs.python.org/issue6769    krisvale                      
                                                                               

UnicodeDecodeError when retrieving binary data from cgi.FieldSto  126 days
       http://bugs.python.org/issue6854    r.david.murray                
                                                                               

test_distutils failure                                              0 days
       http://bugs.python.org/issue6961    flox                          
       buildbot                                                                

tmpnam should not be used if tempfile or mkstemp are available    110 days
       http://bugs.python.org/issue6965    flox                          
                                                                               

test_urllib: unsetting missing 'env' variable                       0 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

distutils and IronPython compatibility                             73 days
       http://bugs.python.org/issue7071    tarek                         
       26backport                                                              

weak dict iterators are fragile because of unpredictable GC runs   89 days
       http://bugs.python.org/issue7105    pitrou                        
       patch                                                                   

email:  msg.items() returns different values before and after ms   89 days
       http://bugs.python.org/issue7119    r.david.murray                
       patch                                                                   

get_payload(decode=True) eats last newline                         87 days
       http://bugs.python.org/issue7143    r.david.murray                
                                                                               

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo   49 days
       http://bugs.python.org/issue7382    alexandre.vassalotti          
       patch, easy                                                             

Document inspect.get(full)argspec limitation to Python function    39 days
       http://bugs.python.org/issue7422    georg.brandl                  
                                                                               

Py3.1: Fatal Python Error: Py_Initialize...unknown encoding: chc   35 days
       http://bugs.python.org/issue7441    georg.brandl                  
                                                                               

when piping output between subprocesses some fd is left open blo   36 days
       http://bugs.python.org/issue7448    flox                          
                                                                               

Solaris SPARC: _multiprocessing.so: symbol sem_timedwait: refere   35 days
       http://bugs.python.org/issue7454    srid                          
                                                                               

cPickle: stack underflow in load_pop()                             33 days
       http://bugs.python.org/issue7455    haypo                         
       patch                                                                   

crash in str.rfind() with an invalid start value                   33 days
       http://bugs.python.org/issue7458    haypo                         
       patch                                                                   

test_multiprocessing test_rapid_restart fails if port 9999 alrea   27 days
       http://bugs.python.org/issue7498    r.david.murray                
       patch, buildbot                                                         

Extended slicing with classic class behaves strangely               3 days
       http://bugs.python.org/issue7532    mark.dickinson                
       patch                                                                   

stringlib fastsearch could be improved on 64-bit builds            13 days
       http://bugs.python.org/issue7607    pitrou                        
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option    7 days
       http://bugs.python.org/issue7617    tarek                         
       patch                                                                   

[patch] improve unicode methods: split() rsplit() and replace()    10 days
       http://bugs.python.org/issue7622    pitrou                        
       patch                                                                   

bytearray needs more tests for "b.some_method()[0] is not b"       10 days
       http://bugs.python.org/issue7625    pitrou                        
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                    4 days
       http://bugs.python.org/issue7648    orsenthil                     
       easy                                                                    

[patch] Enable additional bytes and memoryview tests.               5 days
       http://bugs.python.org/issue7654    pitrou                        
       patch                                                                   

test_ctypes failure on AIX 5.3                                      4 days
       http://bugs.python.org/issue7657    mallayya                      
       patch                                                                   

Two float('nan') are not equal                                      0 days
       http://bugs.python.org/issue7660    mark.dickinson                
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points     1 days
       http://bugs.python.org/issue7663    lemburg                       
                                                                               

python logger does not handle IOError Exception                     1 days
       http://bugs.python.org/issue7664    vinay.sajip                   
                                                                               

test_lib2to3 fails if path contains space                           0 days
       http://bugs.python.org/issue7666    benjamin.peterson             
                                                                               

test_unicode_file fails with non-ascii path                         2 days
       http://bugs.python.org/issue7669    ezio.melotti                  
                                                                               

Incorrect division in [wave.py]                                     1 days
       http://bugs.python.org/issue7681    benjamin.peterson             
       patch, needs review                                                     

Wrong link in HTML documentation                                    0 days
       http://bugs.python.org/issue7683    ezio.melotti                  
                                                                               

minor typo in re docs                                               0 days
       http://bugs.python.org/issue7685    ezio.melotti                  
       patch                                                                   

Wrong PEP number in importlib module manual page                    0 days
       http://bugs.python.org/issue7690    brett.cannon                  
                                                                               

test_bsddb3 files are not always removed when test fails            2 days
       http://bugs.python.org/issue7691    flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero                 0 days
       http://bugs.python.org/issue7692    mark.dickinson                
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes     0 days
       http://bugs.python.org/issue7697    pjenvey                       
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame     0 days
       http://bugs.python.org/issue7698    skip.montanaro                
       patch                                                                   

"-3" flag does not work anymore                                     1 days
       http://bugs.python.org/issue7700    brett.cannon                  
       patch                                                                   

fix output string length for binascii.b2a_uu()                      1 days
       http://bugs.python.org/issue7701    pitrou                        
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE            0 days
       http://bugs.python.org/issue7704    tim_one                       
                                                                               

Support for sending multipart form data                          2452 days
       http://bugs.python.org/issue727898  r.david.murray                
                                                                               

email.MIME*.as_string removes duplicate spaces                   1440 days
       http://bugs.python.org/issue1422094 r.david.murray                
       easy                                                                    

test_parsedate_acceptable_to_time_functions+DST == :-(           1394 days
       http://bugs.python.org/issue1454285 r.david.murray                
                                                                               

email module does not complay with RFC 2046: CRLF issue          1196 days
       http://bugs.python.org/issue1571841 r.david.murray                
                                                                               

email.FeedParser.BufferedSubFile improperly handles "\r\n"        969 days
       http://bugs.python.org/issue1721862 r.david.murray                
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 20 dtoa.c: oversize b in quorem                                      11 days
open    http://bugs.python.org/issue7632   

 18 _ssl module causes segfault                                        5 days
open    http://bugs.python.org/issue7672   

 12 Speed up cPickle's pickling generally                            287 days
open    http://bugs.python.org/issue5683   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    7 days
open    http://bugs.python.org/issue7658   

  8 Fatal error on thread creation in low memory condition            27 days
open    http://bugs.python.org/issue7544   

  8 Direct calls to PyObject_Del/PyObject_DEL are broken for --with  558 days
open    http://bugs.python.org/issue3299   

  7 "-3" flag does not work anymore                                    1 days
closed  http://bugs.python.org/issue7700   

  7 tarfile.extractall can't have unicode extraction path              2 days
open    http://bugs.python.org/issue7693   

  7 test_urllib: unsetting missing 'env' variable                      0 days
closed  http://bugs.python.org/issue7026   

  7 os.path.abspath with unicode argument should call os.getcwdu     542 days
open    http://bugs.python.org/issue3426   




From g.brandl at gmx.net  Sat Jan 16 21:05:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 16 Jan 2010 21:05:46 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<20100113200840.GC14858@phd.pp.ru>
	<319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
Message-ID: <hit67o$7v4$1@ger.gmane.org>

Am 13.01.2010 21:27, schrieb Lennart Regebro:
> On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
>> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>>> What do you need to do in the PYTHONSTARTUP file?
>>> Ten years of Python programming, and I didn't even know it existed. :-)
>>
>>   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.
> 
> Cheese and fries! :-)
> 
> Anyway, I don't see how this could pose any problems, because any user
> advanced enough to do these things will be advanced enough to
> understand what the problem is and fix it so it works under Python 3
> too.

I'd propose adding a bit of text to the environment variables documentation,
and especially about the needs of PYTHONSTARTUP files.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From asmodai at in-nomine.org  Sat Jan 16 21:50:56 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 16 Jan 2010 21:50:56 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <874ompg225.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
Message-ID: <20100116205056.GL99670@nexus.in-nomine.org>

-On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>hehe. tab completion:

With bpython and ipython available, why would you even want to stick to the
'plain old' interactive interpreter?

(Sorry to derail the discussion, but maybe there's more people that have not
heard of either or both.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For ever, brother, hail and farewell...


From solipsis at pitrou.net  Sat Jan 16 21:57:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jan 2010 20:57:13 +0000 (UTC)
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <loom.20100116T215639-769@post.gmane.org>

Jeroen Ruigrok van der Werven <asmodai <at> in-nomine.org> writes:
> 
> -On [20100113 22:13], Ralf Schmitt (ralf <at> brainbot.com) wrote:
> >hehe. tab completion:
> 
> With bpython and ipython available, why would you even want to stick to the
> 'plain old' interactive interpreter?

Why wouldn't we?
There are probably many more people using the standard Python prompt than
ipython.





From ben+python at benfinney.id.au  Sat Jan 16 23:41:03 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2010 09:41:03 +1100
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <87y6jxk70g.fsf@benfinney.id.au>

Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:

> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
> >hehe. tab completion:
>
> With bpython and ipython available, why would you even want to stick
> to the 'plain old' interactive interpreter?

Because those optional extras are not always available, whereas the
standard interactive interpreter is always available with CPython
distributions.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Sun Jan 17 01:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:22:10 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87y6jxk70g.fsf@benfinney.id.au>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<874ompg225.fsf@brainbot.com>	<20100116205056.GL99670@nexus.in-nomine.org>
	<87y6jxk70g.fsf@benfinney.id.au>
Message-ID: <4B525832.8090904@gmail.com>

Ben Finney wrote:
> Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:
> 
>> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>>> hehe. tab completion:
>> With bpython and ipython available, why would you even want to stick
>> to the 'plain old' interactive interpreter?
> 
> Because those optional extras are not always available, whereas the
> standard interactive interpreter is always available with CPython
> distributions.

I've never even contemplated the idea of installing 3rd party apps for
the 4 current development builds (2.x trunk, 2.x maintenance, 3.x trunk,
3.x maintenance) on my home development machine.

And of course work suffers from the problem of not being allowed to
install arbitrary apps I downloaded from the internet without getting
the licensing vetted for commercial acceptability.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 01:34:08 2010
From: nad at acm.org (Ned Deily)
Date: Sat, 16 Jan 2010 16:34:08 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
Message-ID: <nad-79FC16.16340816012010@news.gmane.org>

I've recently seen a couple of references to 3.1.2 go by in checkins 
which made me wonder whether dates have been proposed yet for updates to 
either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
references in the PEPs.  Some advance warning would be nice.  I assume 
that some critical problem could trigger planning for an update on short 
notice; is there a time-limit trigger as well?

-- 
 Ned Deily,
 nad at acm.org



From ncoghlan at gmail.com  Sun Jan 17 01:55:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:55:38 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52600A.7060201@gmail.com>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Usually there is one more regular maintenance release of the existing
maintenance branches within a few months of the creation of the next
version (releases before then are at the discretion of the release
manager, and security releases continue for much longer).

So take the planned 2.7 and 3.2 release dates and add a couple of months
to each one to get the likely release dates for the 2.6.x and 3.1.x
maintenance releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Sun Jan 17 10:21:49 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Jan 2010 10:21:49 +0100
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52D6AD.6090302@v.loewis.de>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Barry was once proposing time-based releases; this idea didn't really
catch on.

It's the release manager who decides when the next bug fix release will
be made, often in response to developers asking for one.

Regards,
Martin



From solipsis at pitrou.net  Sun Jan 17 14:00:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 13:00:04 +0000 (UTC)
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <loom.20100117T135910-313@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.

There are a couple of release blockers right now. Once they are fixed or
deferred, I think it would be nice to have a 3.1.2.
Why do you need "some advance warning" though?




From ncoghlan at gmail.com  Sun Jan 17 14:40:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 23:40:42 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <loom.20100117T135910-313@post.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
Message-ID: <4B53135A.7060104@gmail.com>

Antoine Pitrou wrote:
> Ned Deily <nad <at> acm.org> writes:
>> I've recently seen a couple of references to 3.1.2 go by in
>> checkins which made me wonder whether dates have been proposed yet
>> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
>> didn't see any references in the PEPs.  Some advance warning would
>> be nice.
> 
> There are a couple of release blockers right now. Once they are fixed
> or deferred, I think it would be nice to have a 3.1.2. Why do you
> need "some advance warning" though?

Advance warning does allow interested users that would consider
upgrading to schedule time for testing before the maintenance release
comes out. This is particularly useful in helping to make a 1-week RC
period effective in picking up issues that might otherwise lead to a
brown paper bag release to fix major issues that slipped through our own
automated test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 19:01:37 2010
From: nad at acm.org (Ned Deily)
Date: Sun, 17 Jan 2010 10:01:37 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
	<4B53135A.7060104@gmail.com>
Message-ID: <nad-25165C.10013717012010@ger.gmane.org>

In article <4B53135A.7060104 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> I've recently seen a couple of references to 3.1.2 go by in
> >> checkins which made me wonder whether dates have been proposed yet
> >> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
> >> didn't see any references in the PEPs.  Some advance warning would
> >> be nice.
> > There are a couple of release blockers right now. Once they are fixed
> > or deferred, I think it would be nice to have a 3.1.2. Why do you
> > need "some advance warning" though?
> 
> Advance warning does allow interested users that would consider
> upgrading to schedule time for testing before the maintenance release
> comes out. This is particularly useful in helping to make a 1-week RC
> period effective in picking up issues that might otherwise lead to a
> brown paper bag release to fix major issues that slipped through our own
> automated test coverage.

That. and resource contention: there are always potential fixes in the 
pipeline that could or should be bumped in priority if one knows there 
is a code cutoff approaching.

-- 
 Ned Deily,
 nad at acm.org



From ziade.tarek at gmail.com  Sun Jan 17 20:51:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 20:51:58 +0100
Subject: [Python-Dev] Enhancing the shutil module
Message-ID: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>

Hello,

For 2.7/3.2, I am in the process of removing modules in Distutils that
can be replaced by calls to existing functions in stdlib. For
instance, "dir_util" and "file_util" (old modules from the Python 1.x
era) are going away in favor of calls to shutil (and os), so the
Distutils package gets lighter.

Another module I would like to move away from Distutils is
"archive_util". It contains helpers to build archives, whether they
are zip or tar files. I propose to move those useful functions into
shutil, as this seems the most logical place.

I also propose to maintain this "shutil" module for now on (no one is
declared as a maintainer in maintainers.rst) since Distutils will
become a heavy user of its functions.

Any objections/opinions ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 20:53:03 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 19:53:03 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <4B536A9F.5050203@voidspace.org.uk>

On 17/01/2010 19:51, Tarek Ziad? wrote:
> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>
> Regards,
> Tarek
>
>    
I think it's a great idea. :-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From brett at python.org  Sun Jan 17 20:55:47 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jan 2010 11:55:47 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>

On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>

If it's archive-agnostic then shutil is probably the best place.


>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
>
Great!


> Any objections/opinions ?
>

No objections from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100117/f97ac6eb/attachment-0006.htm>

From solipsis at pitrou.net  Sun Jan 17 21:04:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 20:04:41 +0000 (UTC)
Subject: [Python-Dev] Enhancing the shutil module
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <loom.20100117T210307-719@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.

Are these functions portable? Do they rely on external programs?

> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.

It's ok for me.

Regards

Antoine.




From ziade.tarek at gmail.com  Sun Jan 17 21:09:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:09:18 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
Message-ID: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>

On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Hello,
>>
>> For 2.7/3.2, I am in the process of removing modules in Distutils that
>> can be replaced by calls to existing functions in stdlib. For
>> instance, "dir_util" and "file_util" (old modules from the Python 1.x
>> era) are going away in favor of calls to shutil (and os), so the
>> Distutils package gets lighter.
>>
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>
> If it's archive-agnostic then shutil is probably the best place.

In more details:
It allows the creation of gzip, bzip2, tar and zip files through a single API.
There's a registry of supported formats and the API is driven by a
format identifier.

To do the work it uses stdlib's compression modules. Although it tries
the "zip" system command as a fallback if the "zipfile" module is not
present.

(notice that I've removed the support of "compress" (.Z) some time ago)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 21:15:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 20:15:00 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <loom.20100117T210307-719@post.gmane.org>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
Message-ID: <4B536FC4.9090304@voidspace.org.uk>

On 17/01/2010 20:04, Antoine Pitrou wrote:
> Tarek Ziad?<ziade.tarek<at>  gmail.com>  writes:
>    
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>>      
> Are these functions portable? Do they rely on external programs?
>
>    

I believe that part of the work that Tarek has been doing has been to 
make these distutils commands use the Python standard library and not 
depend on external programs. In which case they seem like *excellent* 
additions to the shutil module.

Of course Tarek can speak for himself...

Michael

>> I also propose to maintain this "shutil" module for now on (no one is
>> declared as a maintainer in maintainers.rst) since Distutils will
>> become a heavy user of its functions.
>>      
> It's ok for me.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Sun Jan 17 21:43:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:43:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B536FC4.9090304@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
Message-ID: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:15 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>> Are these functions portable? Do they rely on external programs?
>>
>>
>
> I believe that part of the work that Tarek has been doing has been to make
> these distutils commands use the Python standard library and not depend on
> external programs. In which case they seem like *excellent* additions to the
> shutil module.

yes, in the past the "tar" files where created using the "tar" command
but this has been changed. For a while now, they are portable and they
use stdlib code only. A recent addition is to be able to define
user/group permissions in the tar files, thanks to Lars' work in the
tarfile module.

There's one remaining external call for "zip" done if the zip module
is not found, but I am happy to remove it and throw an exception if
it's not found, and keep the external "zip" call on Distutils side, so
shutil stays 100% stdlib-powered.

> Of course Tarek can speak for himself...

Thanks for explaining it ! :)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From sridharr at activestate.com  Sun Jan 17 22:50:52 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 17 Jan 2010 13:50:52 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
Message-ID: <4B53863C.5060304@activestate.com>

On 1/17/2010 12:09 PM, Tarek Ziad? wrote:
> On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon<brett at python.org>  wrote:
>> >  On Sun, Jan 17, 2010 at 11:51, Tarek Ziad?<ziade.tarek at gmail.com>  wrote:
>>> >>  Another module I would like to move away from Distutils is
>>> >>  "archive_util". It contains helpers to build archives, whether they
>>> >>  are zip or tar files. I propose to move those useful functions into
>>> >>  shutil, as this seems the most logical place.
>> >  If it's archive-agnostic then shutil is probably the best place.
> In more details:
> It allows the creation of gzip, bzip2, tar and zip files through a single API.
> There's a registry of supported formats and the API is driven by a
> format identifier.

Will it also allow decompression of the said archive types? Distribute 
has some utility code to handle zip/tar archives. So does PyPM. This is 
because the `tarfile` and `zipfile` modules do not "just work" due to 
several issues.

See http://gist.github.com/279606

Take note of the following in the above code:

  1) _ensure_read_write_access
  2) *File.is_valid
  3) ZippedFile.extract ... issue 6510
  4) ZippedFile.extract ... issue 6609
  5) TarredFile.extract ... issue 6584
  6) The way unpack() detects the unpacked directory.

-srid


From ziade.tarek at gmail.com  Sun Jan 17 23:09:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 23:09:29 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B53863C.5060304@activestate.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
	<4B53863C.5060304@activestate.com>
Message-ID: <94bdd2611001171409j4ec1e0bfj47e59894fdec0d8a@mail.gmail.com>

On Sun, Jan 17, 2010 at 10:50 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
[..]
> Will it also allow decompression of the said archive types?

No but it would make sense having this one as well.
Distribute/Setuptools' "unpack_archive" (used by easy_install) was
implemented using the same principle as Distutils' "make_archive".

I can add it as well while I am at it : it has been successfully used
for years by easy_install.

> Distribute has
> some utility code to handle zip/tar archives. So does PyPM. This is because
> the `tarfile` and `zipfile` modules do not "just work" due to several
> issues.
>
> See http://gist.github.com/279606
>
> Take note of the following in the above code:
>
> ?1) _ensure_read_write_access
> ?2) *File.is_valid
> ?3) ZippedFile.extract ... issue 6510
> ?4) ZippedFile.extract ... issue 6609
> ?5) TarredFile.extract ... issue 6584

Looks like some of these are already fixed (I just looked quickly in
the bug tracker though)
If its not already done and pending, it would be great if you could
refactor your fixes into patches for the remaining issues for each one
of those modules

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From barry at python.org  Sun Jan 17 23:56:37 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Jan 2010 17:56:37 -0500
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <4B52D6AD.6090302@v.loewis.de>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<4B52D6AD.6090302@v.loewis.de>
Message-ID: <BEC881FD-2BDE-4B5C-A570-B8C1E23D6DE1@python.org>

On Jan 17, 2010, at 4:21 AM, Martin v. L?wis wrote:

> Barry was once proposing time-based releases; this idea didn't really
> catch on.

I'm still a proponent of this, but it doesn't seem like there's enough support for it.

> It's the release manager who decides when the next bug fix release will
> be made, often in response to developers asking for one.

I'm happy to make a 2.6.5 release whenever it makes sense.
-Barry



From ncoghlan at gmail.com  Mon Jan 18 13:40:01 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Jan 2010 22:40:01 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
Message-ID: <4B5456A1.2080109@gmail.com>

Tarek Ziad? wrote:
> There's one remaining external call for "zip" done if the zip module
> is not found, but I am happy to remove it and throw an exception if
> it's not found, and keep the external "zip" call on Distutils side, so
> shutil stays 100% stdlib-powered.

+1 for that approach. These changes all sound like nice additions to
shutil, and hooray for every module that gets adopted by an active
maintainer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Mon Jan 18 14:34:14 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 14:34:14 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5456A1.2080109@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
Message-ID: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>

On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
> 
> Tarek Ziad? wrote:
>> There's one remaining external call for "zip" done if the zip module
>> is not found, but I am happy to remove it and throw an exception if
>> it's not found, and keep the external "zip" call on Distutils side, so
>> shutil stays 100% stdlib-powered.
> 
> +1 for that approach. These changes all sound like nice additions to
> shutil, and hooray for every module that gets adopted by an active
> maintainer :)

Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).

Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

From doug.hellmann at gmail.com  Mon Jan 18 14:46:05 2010
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 18 Jan 2010 08:46:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils  
>>> side, so
>>> shutil stays 100% stdlib-powered.
>>
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
>
> Isn't it a bit weird to include that to shutil though? shutil  
> advertises itself as "a number of high-level operations on files and  
> collections of files." and from what I understood it was a bunch of  
> shell-type utility functions to easily copy, move or remove files  
> and directories (that's pretty much all there is in it at this time).
>
> Wouldn't it make more sense to put those "archive utils" functions/ 
> objects in a new module separate from shutil, dealing specifically  
> with cross-archive APIs and linked from the current archive-specific  
> modules (essentially, just take the current archive_util, move it to  
> the toplevel of the stdlib and maybe rename it)? It would also make  
> the module much easier to find when searching through the module  
> listing, I think.

+1



From fuzzyman at voidspace.org.uk  Mon Jan 18 14:57:49 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 18 Jan 2010 13:57:49 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
Message-ID: <4B5468DD.5040200@voidspace.org.uk>

On 18/01/2010 13:46, Doug Hellmann wrote:
>
> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>
>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>
>>> Tarek Ziad? wrote:
>>>> There's one remaining external call for "zip" done if the zip module
>>>> is not found, but I am happy to remove it and throw an exception if
>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>> shutil stays 100% stdlib-powered.
>>>
>>> +1 for that approach. These changes all sound like nice additions to
>>> shutil, and hooray for every module that gets adopted by an active
>>> maintainer :)
>>
>> Isn't it a bit weird to include that to shutil though? shutil 
>> advertises itself as "a number of high-level operations on files and 
>> collections of files." 

Well - isn't what's being proposed "a number of high-level operations on 
files and collections of files." ?

>> and from what I understood it was a bunch of shell-type utility functions

like tar and zip? :-)

>> to easily copy, move or remove files and directories (that's pretty 
>> much all there is in it at this time).
>>

I don't think the additions are out of place prima-facie.

>> Wouldn't it make more sense to put those "archive utils" 
>> functions/objects in a new module separate from shutil, dealing 
>> specifically with cross-archive APIs and linked from the current 
>> archive-specific modules (essentially, just take the current 
>> archive_util, move it to the toplevel of the stdlib and maybe rename 
>> it)? It would also make the module much easier to find when searching 
>> through the module listing, I think.
>
> +1
>

Proliferation of modules is itself a bad thing though.

Michael


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Mon Jan 18 15:34:12 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 15:34:12 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <94bdd2611001180634sacd998fm6f67dc680e2e61c3@mail.gmail.com>

On Mon, Jan 18, 2010 at 2:57 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>>> Wouldn't it make more sense to put those "archive utils"
>>> functions/objects in a new module separate from shutil, dealing specifically
>>> with cross-archive APIs and linked from the current archive-specific modules
>>> (essentially, just take the current archive_util, move it to the toplevel of
>>> the stdlib and maybe rename it)? It would also make the module much easier
>>> to find when searching through the module listing, I think.
>>
>> +1
>>
>
> Proliferation of modules is itself a bad thing though.

I am with Michael here. I think having this function in shutil is the
right place.

For the find problem, I think docs.python.org is easy to search now,
as long as the shutil module documentation has an good set of examples
for this new API.

We could even add references in the tarfile, zipfile modules
documentation pointing to these examples.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org


From masklinn at masklinn.net  Mon Jan 18 15:56:05 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 15:56:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>

On 18 Jan 2010, at 14:57 , Michael Foord wrote:
> 
> On 18/01/2010 13:46, Doug Hellmann wrote:
>> 
>> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>> 
>>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>> 
>>>> Tarek Ziad? wrote:
>>>>> There's one remaining external call for "zip" done if the zip module
>>>>> is not found, but I am happy to remove it and throw an exception if
>>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>>> shutil stays 100% stdlib-powered.
>>>> 
>>>> +1 for that approach. These changes all sound like nice additions to
>>>> shutil, and hooray for every module that gets adopted by an active
>>>> maintainer :)
>>> 
>>> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." 
> 
> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
> 
Well no those are high-level operations on a very restricted set of file types (archives).

>>> and from what I understood it was a bunch of shell-type utility functions
> 
> like tar and zip? :-)
> 
Tar and zip have a module each at this time, so they're considered pretty big. I don't think anybody would consider "mv" big enough to give it a module on its own.

>>> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.
>> 
>> +1
>> 
> 
> Proliferation of modules is itself a bad thing though.
Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.

Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

From phd at phd.pp.ru  Mon Jan 18 16:03:38 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 18 Jan 2010 18:03:38 +0300
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <20100118150337.GA19391@phd.pp.ru>

On Mon, Jan 18, 2010 at 02:34:14PM +0100, Masklinn wrote:
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

   +1

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From ziade.tarek at gmail.com  Mon Jan 18 16:24:37 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 16:24:37 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
	<6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
Message-ID: <94bdd2611001180724u7f48e620ub32e13ef96c873b9@mail.gmail.com>

On Mon, Jan 18, 2010 at 3:56 PM, Masklinn <masklinn at masklinn.net> wrote:
[..]
>> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
>>
> Well no those are high-level operations on a very restricted set of file types (archives)

not really: make_archive/unpack_archive, are also dealing with files
and directories.

[..]
>> Proliferation of modules is itself a bad thing though.
> Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.
>
> Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

I am not sure why this would happen. For instance, shutil is already
on the top of the os module since a few major versions IIRC, because
it reads and writes files and directories. But it was not moved into
the os package (or vice-versa)

The shutil module uses APIs to read and write files. So if it works
with archives, it's just a specific read/write API that is used, but
that doesn't mean tarfile and zipfile might be reunited with shutil
imho.

If the shutil module is restricted to high-level files and directories
manipulation, working with archives is just a target like another I
think.

But at the end I am 0- to create a new module, because what really
matters to me is to take it out of Distutils :)

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From listsin at integrateddevcorp.com  Mon Jan 18 16:56:05 2010
From: listsin at integrateddevcorp.com (Steve Steiner (listsin))
Date: Mon, 18 Jan 2010 10:56:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>> 
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils side, so
>>> shutil stays 100% stdlib-powered.
>> 
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
> 
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

As much of a pain as it is to get new modules accepted, I agree that mixing archiving functions into shutil is not the right way to do it and that a separate archive_util module would make much more sense and would give a logical place to put any extensions to archive handling.

S





From rdmurray at bitdance.com  Mon Jan 18 20:28:47 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 18 Jan 2010 14:28:47 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
Message-ID: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>

On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
> As much of a pain as it is to get new modules accepted, I agree that
> mixing archiving functions into shutil is not the right way to do it
> and that a separate archive_util module would make much more sense and
> would give a logical place to put any extensions to archive handling.

Looking at the source code and API for both shutil and archive_util, I
think that the archive_util methods fit into shutil.  shutil currently
wraps some standard library facilities with convenience functions
for operations you might otherwise perform at the shell command line using
OS facilities.  As far as I can tell, archive_util does the same, and
seems quite within the shutil mission of "high level file operations".

So +1 from me for putting these in shutil.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From p.f.moore at gmail.com  Mon Jan 18 20:48:43 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 18 Jan 2010 19:48:43 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
Message-ID: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>

2010/1/18 R. David Murray <rdmurray at bitdance.com>:
> On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
>> As much of a pain as it is to get new modules accepted, I agree that
>> mixing archiving functions into shutil is not the right way to do it
>> and that a separate archive_util module would make much more sense and
>> would give a logical place to put any extensions to archive handling.
>
> Looking at the source code and API for both shutil and archive_util, I
> think that the archive_util methods fit into shutil. ?shutil currently
> wraps some standard library facilities with convenience functions
> for operations you might otherwise perform at the shell command line using
> OS facilities. ?As far as I can tell, archive_util does the same, and
> seems quite within the shutil mission of "high level file operations".
>
> So +1 from me for putting these in shutil.

Conceptually, I'm happy with these going into shutil (and +1 on the
rest of Tarek's proposal, too!)

To my mind, shutil is a module for higher-level operations on files -
the sort of things you'd do in shell commands, like move a batch of
files around (mv), create a directory tree (mkdir -p). Tarring or
zipping up a batch of files fits nicely into that space.

Paul.


From david.lyon at preisshare.net  Tue Jan 19 02:53:43 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 Jan 2010 20:53:43 -0500
Subject: [Python-Dev] PyCon Keynote
In-Reply-To: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
References: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
Message-ID: <0dfb370163cf3a284ca256f49662db74@preisshare.net>

On Wed, 13 Jan 2010 10:51:14 -0800, Guido van Rossum <guido at python.org>
wrote:
> Please mail me topics you'd like to hear me talk about in my keynote
> at PyCon this year.

As a typical (but perphaps more vocal) python developer I would
like to hear things that might aspire me and others to move to 
python 3.

The way I see it is that python 2.x represents everyones 
comfort zone. It's safe and nobody got fired at work for 
using python 2.x

I would like to hear how python 3 could be 'shaken' up
with a slightly less conservative policy that has gone
with the python 2.x series.

The logic perhaps being that since only a minority
use the 3.x series, it's functionality set is still
more up in the air. imo it needs bigger batteries
to give it more power than the 2.x series.

This meaning that the stdlib could take an extra
5-10 packages not in the 2.x series. Just as
one example, sphinx - a great documentation
tool. I can easily name five or six others.

I'd also like to hear more of your ideas on pypi
and getting it to have as much who-ha as CPAN.

You could do a lot to enlarge the developer
group. Help us all get our priorities to be
inline with your own wishes and expectations.

If you ask us all to put in a big year and
buy you a beer at the end.. I'm certain we
all will.

Every little bit of inspiration and direction
counts for a lot...

Best Regards

David











From ncoghlan at gmail.com  Tue Jan 19 12:20:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Jan 2010 21:20:05 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
	<79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
Message-ID: <4B559565.8090602@gmail.com>

Paul Moore wrote:
> 2010/1/18 R. David Murray <rdmurray at bitdance.com>:
>> So +1 from me for putting these in shutil.
> 
> Conceptually, I'm happy with these going into shutil (and +1 on the
> rest of Tarek's proposal, too!)
> 
> To my mind, shutil is a module for higher-level operations on files -
> the sort of things you'd do in shell commands, like move a batch of
> files around (mv), create a directory tree (mkdir -p). Tarring or
> zipping up a batch of files fits nicely into that space.

This is also reflected in the way at least Windows handles archives
these days - it took them a couple of iterations to get it right (and
resolve some of the performance impacts), but Explorer now does a decent
job of integrating archives into the directory tree as "folders that
happen to be compressed".

Are archives as fundamental as directories and files? No. But in the
context of shutil, the fact that their internal structure is largely
about directories and files makes them more than just another arbitrary
file type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 19 14:16:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 08:16:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
Message-ID: <20100119081639.5d431ed9@freewill>

I've just updated the Launchpad mirrors for the 4 active Python branches,
trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar branches
on code.python.org but it's probably been 7 months or so since those were
regularly updated.  Now the Launchpad branches sync against the read-only
Subversion branches at http://svn.python.org, so they should remain up-to-date
(within the re-sync timeframe of about 4 hours).

This means you can once again use Bazaar to get local branches of Python, and
you can of course push your own branches to Launchpad.  I believe you can even
use the bzr-svn plugin to commit changes back to the Subversion master, though
I have not yet tried this.

To get a local branch, just do any of the following:

    % bzr branch lp:python (for trunk)
    % bzr branch lp:python/2.6
    % bzr branch lp:python/py3k
    % bzr branch lp:python/3.1

(It's fairly easy to create new mirrors for other Subversion branches,
e.g. Python 2.5; just drop me an email if you want them.)

If you're going to create a lot of branches you probably want to put them in a
shared repository.  E.g.

    % bzr init-repo pythonbzr
    % cd pythonbzr
    % bzr branch lp:python/py3k

Bazaar 2.0 or better is recommended.  For me, it took about 5m to check the
first branch out from Launchpad, and then about 30s or so for each subsequent
branch.

Enjoy,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/89a42d75/attachment-0004.pgp>

From abpillai at gmail.com  Tue Jan 19 14:37:18 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Tue, 19 Jan 2010 19:07:18 +0530
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <8548c5f31001190537l2bcab5cfkb6f461910e4abd8b@mail.gmail.com>

On Mon, Jan 18, 2010 at 1:21 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>

+1 for this. Just make sure that you change the docstring of shutil
 which now reads as,

" shutil - Utility functions for copying files and directory trees."

 According to this "definition", archives don't fit in there. But the
 functionality does fit right in, so just need to make sure that it
 is reflected in the __doc__ .


> Regards,
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/55892759/attachment-0004.htm>

From vinay_sajip at yahoo.co.uk  Tue Jan 19 16:50:42 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 19 Jan 2010 15:50:42 +0000 (UTC)
Subject: [Python-Dev] Mailing List archive corruption?
Message-ID: <loom.20100119T164757-651@post.gmane.org>

Hi,

When I look at the mailing list archive for python-dev, I see some odd stuff at
the bottom of the page:

http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232

Anyone know what's happened?

Regards,

Vinay Sajip



From barry at python.org  Tue Jan 19 17:07:48 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:07:48 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <loom.20100119T164757-651@post.gmane.org>
References: <loom.20100119T164757-651@post.gmane.org>
Message-ID: <20100119110748.69bc564a@freewill>

On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:

>When I look at the mailing list archive for python-dev, I see some odd stuff at
>the bottom of the page:
>
>http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>
>Anyone know what's happened?

WTF?  I think the archives were recently regenerated, so there's probably a
fubar there.  CC'ing the postmasters.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/8947a2e9/attachment-0004.pgp>

From foom at fuhm.net  Tue Jan 19 17:24:57 2010
From: foom at fuhm.net (James Y Knight)
Date: Tue, 19 Jan 2010 11:24:57 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119110748.69bc564a@freewill>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
Message-ID: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>


On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:

> On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
>
>> When I look at the mailing list archive for python-dev, I see some  
>> odd stuff at
>> the bottom of the page:
>>
>> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>>
>> Anyone know what's happened?
>
> WTF?  I think the archives were recently regenerated, so there's  
> probably a
> fubar there.  CC'ing the postmasters.

That happens if messages had unescaped "From" lines in the middle of  
them.

No doubt, you've now broken every link anyone had ever made into the  
python-dev archives, because now all the article numbers are  
different. BTDT...unfortunately... Pipermail really is quite crappy,  
sigh.

Anyhow, when I did that, I went back to a backup to get the original  
article numbers, and edited the mbox file escaping From lines or  
adding additional empty messages until the newly regenerated article  
numbers matched the originals. I'd highly recommend going through that  
painful process, since I suspect a *lot* of people have links to the  
python-dev archive. Hope you have a backup (or can find caches on  
google or archive.org or something).

James


From barry at python.org  Tue Jan 19 17:44:21 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:44:21 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119114421.3b96fbd4@freewill>

On Jan 19, 2010, at 11:24 AM, James Y Knight wrote:

>No doubt, you've now broken every link anyone had ever made into the  
>python-dev archives, because now all the article numbers are  
>different. BTDT...unfortunately... Pipermail really is quite crappy,  
>sigh.

I've been trying for 10+ years to get folks interested in helping me fix this
(and a few other warts) in Pipermail, to not much success. ;/

>Anyhow, when I did that, I went back to a backup to get the original  
>article numbers, and edited the mbox file escaping From lines or  
>adding additional empty messages until the newly regenerated article  
>numbers matched the originals. I'd highly recommend going through that  
>painful process, since I suspect a *lot* of people have links to the  
>python-dev archive. Hope you have a backup (or can find caches on  
>google or archive.org or something).

bin/cleanarch uses a set of heuristics to find unescaped From lines and fix
them.  It's generally pretty good, but it certain can change message numbers
(and sadly, their urls).

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/e32aa882/attachment-0004.pgp>

From rdmurray at bitdance.com  Tue Jan 19 18:48:41 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Jan 2010 12:48:41 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119174841.9BC3B16A53@kimball.webabinitio.net>

On Tue, 19 Jan 2010 11:24:57 -0500, James Y Knight <foom at fuhm.net> wrote:
> 
> On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:
> 
> > On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
> >
> >> When I look at the mailing list archive for python-dev, I see some  
> >> odd stuff at the bottom of the page:
> >>
> >> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
> >>
> >> Anyone know what's happened?
> >
> > WTF?  I think the archives were recently regenerated, so there's  
> > probably a fubar there.  CC'ing the postmasters.
> 
> That happens if messages had unescaped "From" lines in the middle of  
> them.
> 
> No doubt, you've now broken every link anyone had ever made into the  
> python-dev archives, because now all the article numbers are  
> different. BTDT...unfortunately... Pipermail really is quite crappy,  
> sigh.
> 
> Anyhow, when I did that, I went back to a backup to get the original  
> article numbers, and edited the mbox file escaping From lines or  
> adding additional empty messages until the newly regenerated article  
> numbers matched the originals. I'd highly recommend going through that  
> painful process, since I suspect a *lot* of people have links to the  
> python-dev archive. Hope you have a backup (or can find caches on  
> google or archive.org or something).

The Python issue tracker does, for one.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From ncoghlan at gmail.com  Tue Jan 19 22:18:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 07:18:52 +1000
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119174841.9BC3B16A53@kimball.webabinitio.net>
References: <loom.20100119T164757-651@post.gmane.org>	<20100119110748.69bc564a@freewill>	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
	<20100119174841.9BC3B16A53@kimball.webabinitio.net>
Message-ID: <4B5621BC.4070608@gmail.com>

R. David Murray wrote:
> The Python issue tracker does, for one.

And all the PEPs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From david.lyon at pythontest.org  Wed Jan 20 00:16:44 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 10:16:44 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119081639.5d431ed9@freewill>
References: <20100119081639.5d431ed9@freewill>
Message-ID: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>


Hi Barry,

That looks very interesting...

So does that mean we could update the stdlib for a given
python version using this ?

David

> I've just updated the Launchpad mirrors for the 4 active Python branches,
> trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar
> branches
> on code.python.org but it's probably been 7 months or so since those were
> regularly updated.  Now the Launchpad branches sync against the read-only
> Subversion branches at http://svn.python.org, so they should remain
> up-to-date
> (within the re-sync timeframe of about 4 hours).
>
> This means you can once again use Bazaar to get local branches of Python,
> and
> you can of course push your own branches to Launchpad.  I believe you can
> even
> use the bzr-svn plugin to commit changes back to the Subversion master,
> though
> I have not yet tried this.
>
> To get a local branch, just do any of the following:
>
>     % bzr branch lp:python (for trunk)
>     % bzr branch lp:python/2.6
>     % bzr branch lp:python/py3k
>     % bzr branch lp:python/3.1
>
> (It's fairly easy to create new mirrors for other Subversion branches,
> e.g. Python 2.5; just drop me an email if you want them.)
>
> If you're going to create a lot of branches you probably want to put them
> in a
> shared repository.  E.g.
>
>     % bzr init-repo pythonbzr
>     % cd pythonbzr
>     % bzr branch lp:python/py3k
>
> Bazaar 2.0 or better is recommended.  For me, it took about 5m to check
> the
> first branch out from Launchpad, and then about 30s or so for each
> subsequent
> branch.
>
> Enjoy,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/david.lyon%40pythontest.org
>




From barry at python.org  Wed Jan 20 00:54:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 18:54:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119185439.299660c7@freewill>

On Jan 20, 2010, at 10:16 AM, David Lyon wrote:

>Hi Barry,
>
>That looks very interesting...

Hi David,

>So does that mean we could update the stdlib for a given
>python version using this ?

In a sense, yes (if I understand your question correctly).

You can use Bazaar to branch any of the 4 Python series and use all the modern
DVCS goodness to develop your updates.  You can share your branches with
others via e.g. Launchpad and even request reviews (called "merge proposals")
to get feedback from others.

The one thing I am unsure about, mostly because I have not tried it, is
whether your Bazaar branch can be used to commit directly back to the Python
Subversion master branches.  I /think/ the answer is yes, assuming of course
that you have permission to do so, and that you have a modern version of
Bazaar and the bzr-svn plugin.

It might even be possible to commit your Bazaar branch to a local Subversion
branch, and then commit the latter to get it pushed up to svn.python.org.

At worst, you would use Bazaar's features to get your patch into a state you
and your reviewers are happy with, then you would generate a diff for
application to your copy of the svn.python.org branch.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/37753b1a/attachment-0004.pgp>

From david.lyon at pythontest.org  Wed Jan 20 01:51:23 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 11:51:23 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119185439.299660c7@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
Message-ID: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>

> On Jan 20, 2010, at 10:16 AM, Barry wrote:

>> So does that mean we could update the stdlib for a given
>> python version using this ?
>
> In a sense, yes (if I understand your question correctly).

Yeah, it just needs an implementation.

> The one thing I am unsure about, mostly because I have not tried it, is
> whether your Bazaar branch can be used to commit directly back to the
> Python Subversion master branches.  I /think/ the answer is yes,
> assuming of course that you have permission to do so...

Well I'm too Senior and my stuff is too forward looking to qualify
for that just yet.

I'd be happy to see bzr and mercurial and git all made it together
into the stdlib for python 3. That would give a superb updating
mechanism for python that would propel python well beyond
the dinosaur badlands of CPAN and other languages.

I was actually reading from
(http://en.wikipedia.org/wiki/Python_%28programming_language%29):

"Rather than requiring all desired functionality to be built into the
language's core, Python was designed to be highly extensible. .. .. This
design of a small core language with a large standard library and an
easily extensible interpreter was intended by Van Rossum from the very
start because of his frustrations with ABC (which espoused the opposite
mindset).[5]"

To me, the source code control systems seem to be fully in tune
with the original design of python. That is, to be able to
easily pull external libraries in.

I think what has changed is that the mechanisms now (the SCMs)
are way more highly developed than before. Apart from that
though, after reading the full wikipedia article I'm left
with the distinct impression that things are still pretty
much the same (in that python design philosophy is advanced),
just that the landscape (of external C libraries) has changed.

Now all the libraries are external (on the internet) and
all externally managed.

So with just a tiny amount of work, imho we could pull
it all together to bring python 3 *back* to being that
cool tool that it once was (not saying it isn't now).

Were you offering me an experimental branch somewhere
for python 3 SCM integration ?

David







From jnoller at gmail.com  Wed Jan 20 02:09:15 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 20:09:15 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>

On Tue, Jan 19, 2010 at 7:51 PM, David Lyon <david.lyon at pythontest.org> wrote:
>> On Jan 20, 2010, at 10:16 AM, Barry wrote:
>
>>> So does that mean we could update the stdlib for a given
>>> python version using this ?
>>
>> In a sense, yes (if I understand your question correctly).
>
> Yeah, it just needs an implementation.
>
>> The one thing I am unsure about, mostly because I have not tried it, is
>> whether your Bazaar branch can be used to commit directly back to the
>> Python Subversion master branches. ?I /think/ the answer is yes,
>> assuming of course that you have permission to do so...
>
> Well I'm too Senior and my stuff is too forward looking to qualify
> for that just yet.
>
> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

i sincerely doubt that a source control system will be included in the
standard library in the future. Especially 3. A SCM is not a "package
management system".

Barry was talking about mirrors of the python code. It is true a
"package manager" could be developed based on a SCM, however you need
to implement this far away from the stdlib and get traction with it
within the community long before inclusion would be considered.

The decision to move python's source control from SVN to mercurial was
controversial enough; including 3 or more scm libraries into core
would be an intractable uphill mountain of bike sheds.

> So with just a tiny amount of work, imho we could pull
> it all together to bring python 3 *back* to being that
> cool tool that it once was (not saying it isn't now).

Python 3 is still modularized, still has a standard library, etc. If
you're really interested in helping with the standard library, get on
stdlib-sig, and get ready to write code and PEPs.

> Were you offering me an experimental branch somewhere
> for python 3 SCM integration ?

Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

jesse


From barry at python.org  Wed Jan 20 04:32:30 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 22:32:30 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <20100119223230.4c4a7ed5@freewill>

On Jan 19, 2010, at 08:09 PM, Jesse Noller wrote:

>The decision to move python's source control from SVN to mercurial was
>controversial enough; including 3 or more scm libraries into core
>would be an intractable uphill mountain of bike sheds.

I'd be surprised if any of the big 3 DVCS developers would actually /want/
their stuff in the stdlib.  Being in the stdlib has its advantages and
disadvantages.  I think for rapidly developing technology, the latter can
actually outweigh the former.

(Besides, git in the stdlib doesn't make much sense :).

>Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

Bingo.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/9ef0ed2b/attachment-0004.pgp>

From david.lyon at pythontest.org  Wed Jan 20 04:43:12 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 14:43:12 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>


> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

> Python 3 is still modularized, still has a standard library, etc. If
> you're really interested in helping with the standard library, get on
> stdlib-sig, and get ready to write code and PEPs.

Thank you for your direction to move these items forward to PEPs
and Code.

> i sincerely doubt that a source control system will be included in the
> standard library in the future. Especially 3.

Yeah and who twenty years ago thought you would get a 1GB memory
card for $3 when all we had was 10Meg hard disks and they were
the full 8" platter.

> A SCM is not a "package management system".

Exactly. It almost makes the need for a "package management system"
pretty much obsolete if you can update your code directly from
the developers sources.

That's what all these SCMs provide. Plus it's addictive. It's
hard to go back to 'package' style technology once you have
all your code on an SCM based feed.

> Barry was talking about mirrors of the python code. It is true a
> "package manager" could be developed based on a SCM, however you need
> to implement this far away from the stdlib and get traction with it
> within the community long before inclusion would be considered.

I think I'll have better chances with PEPs.

Being honest, if wonderful libraries like Sphinx and Mercurial
and Git and BZR can't make it into the stdlib, then there is
no hope for even newer code to get in there.

Plus, promoting all sorts of new and fangled tools however
good they may or may not be just confuses users and ends
up being a waste of time imho. It isn't good management
of volounteers time and effort.

If you could imagine disaster relief coordinated this
way, it would just be a disaster in itself.

That's why it has taken some 5 years to get PEP-345 done.

> The decision to move python's source control from SVN to mercurial was
> controversial enough; including 3 or more scm libraries into core
> would be an intractable uphill mountain of bike sheds.

Not at all.

It would be a very fair thing to do. Not to mention being
great for users.

> Barry made bzr mirrors of the python svn tree. Not a python with bzr
> included.

I can't resist asking for that again.. I heard it only in Monty
speek. Did you just say ?:

 "Barry made a bizarre mirror of the python suvern tree. Not a
  python with a buzzer included."

Anyway.. Maybe I do get what your talking about. Even if you do
talk with a strange accent. :-)

David






From jnoller at gmail.com  Wed Jan 20 05:10:07 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 23:10:07 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001192010v66b728dbxa7ad1ed1fc1df15f@mail.gmail.com>

On Tue, Jan 19, 2010 at 10:43 PM, David Lyon <david.lyon at pythontest.org> wrote:
[snip]
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.

Did you ever stop to think that some package authors do not want their
code in the standard library? That throwing random shiny things in
there just makes it a junk drawer?

Besides: Show us the PEP to include Sphinx, and it's dependencies in
the standard lib, with Georg's signature at the bottom.

The authors of modules have to want things to be in there, they have
to be best of breed, tested or common-enough patterns to warrant a
slot. We have things in the standard lib which still need more TLC and
maintenance, or they need to be shunted into space.

Adding random things, which may or may not help packaging, and
installing things from random places in someone source repository
won't make things better.

> Plus, promoting all sorts of new and fangled tools however
> good they may or may not be just confuses users and ends
> up being a waste of time imho. It isn't good management
> of volounteers time and effort.
>
> If you could imagine disaster relief coordinated this
> way, it would just be a disaster in itself.
>
> That's why it has taken some 5 years to get PEP-345 done.

I'm going to assume that you're trolling now, or intentionally
misrepresenting facts. Maybe a little of both. A PEP, and an
implementation and the ability to rationally debate, discuss and
defend your proposal is what is needed to enact changes on policies,
python-core or the standard library.

>> The decision to move python's source control from SVN to mercurial was
>> controversial enough; including 3 or more scm libraries into core
>> would be an intractable uphill mountain of bike sheds.
>
> Not at all.
>
> It would be a very fair thing to do. Not to mention being
> great for users.

There should be one-- and preferably only one --obvious way to do it.

> Anyway.. Maybe I do get what your talking about. Even if you do
> talk with a strange accent. :-)

My sense of humor has been disabled by repeated stunning at your
hands. I admire your enthusiasm, even if I do think some of it is
misplaced, or at guided into the proper channels at very least.

Please, you seem to have the time and willingness to help, please go
about this the right way. Discuss things on the proper lists, make
concrete proposals. If you have have standard lib changes, discuss
them on stdlib-sig, if you have ideas about python-the-language, or
the interpreter, etc - please discuss it on python-ideas.

jesse


From ben+python at benfinney.id.au  Wed Jan 20 05:16:25 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 15:16:25 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87wrzdfm1y.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
> and BZR can't make it into the stdlib, then there is no hope for even
> newer code to get in there.

Those are applications, not libraries. Applications don't belong in the
standard library.

-- 
 \     ?If you pick up a starving dog and make him prosperous, he will |
  `\      not bite you. This is the principal difference between a dog |
_o__)                    and a man.? ?Mark Twain, _Pudd'n'head Wilson_ |
Ben Finney



From brian.curtin at gmail.com  Wed Jan 20 05:19:47 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 19 Jan 2010 22:19:47 -0600
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <cf9f31f21001192019p25665318t3d99caf3db27198e@mail.gmail.com>

On Tue, Jan 19, 2010 at 21:43, David Lyon <david.lyon at pythontest.org> wrote:

>
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
>

I'm not entirely sure I see why the inclusion of a SCM into the stdlib is
necessary.

Just because pieces of software are mature and proven in their fields
doesn't mean we should add them, or that them *not* being in the stdlib
should be a basis for other projects making it in.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/3a2d80f8/attachment-0004.htm>

From barry at python.org  Wed Jan 20 05:26:44 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:26:44 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119232644.7fa25691@freewill>

On Jan 20, 2010, at 02:43 PM, David Lyon wrote:

>> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

>> A SCM is not a "package management system".
>
>Exactly. It almost makes the need for a "package management system"
>pretty much obsolete if you can update your code directly from
>the developers sources.
>
>That's what all these SCMs provide. Plus it's addictive. It's
>hard to go back to 'package' style technology once you have
>all your code on an SCM based feed.

Well...  I'm not so sure.  A package management system like apt does a /ton/
of additional bookkeeping and work to ensure a robust, highly consistent,
functioning system.  And while both Python and most Linux distributions have
their own notion of "package management", they don't always play nicely
together.  Tarek and the distutils-sig's work is trying to make the world a
better place by bridging this gap better, and there is code out there that
makes it easier to say import a Python package from the Cheeseshop and
.deb-ify it for use on Debian and Ubuntu.

There's also work being done in Launchpad that will allow you to
"build-from-branch" so that in a sense you could let a build farm take your
Bazaar branches and automatically build the packages from them.

I've strayed off-topic I suppose, but I see SCMs and package managers as
complementary technologies that help with important parts of the process of
delivering software to end-users, but I don't quite see how one can make the
other obsolete.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/4be8f756/attachment-0004.pgp>

From david.lyon at pythontest.org  Wed Jan 20 05:29:34 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 15:29:34 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>

> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>
> I'd be surprised if any of the big 3 DVCS developers would actually /want/
> their stuff in the stdlib.

If they ask, they'll get told they're motorbike-shedding. "It's better
if their users ask". So here I am as a user doing things the 'right'
way.

> Being in the stdlib has its advantages and
> disadvantages.  I think for rapidly developing technology, the
> latter can actually outweigh the former.

If it's about being able to do updates, then I think this
resolves an old and circular argument. As the SCM implementation
would, one would expect, to be able to update itself.

Side benefits are that it can update everything else along
with it at the same time. User Apps, Packages, whatever.

It's even better having SCM in an Industrial/Scientific
environment. Here's an example:

 - a machine breaks..  (I mean the software for/in it)

 - you fix the code, maybe on the spot

 - you commit and push back to the repository

 - your code gets checked in and run through the testbot
   and then you get blamed and have to do the whole thing
   again properly with a test case. Oh well..

Well anyway, whatever you guys might say, that's a whole
lot more efficient than running back to the development
machine and going through some obscure build and test
and publish process to do a fix on a production machine.

Point : The fact that SCMs are two way is great in
        a production environment. No packaging solution
        can come close.

So why not have python SCMs included as batteries in python..

All these arguments I can take off to the stdlib list when I get
the chance..

David




From barry at python.org  Wed Jan 20 05:50:36 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:50:36 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119235036.57f6e867@freewill>

Okay, last follow up on this and then I'm going to bed. :)

On Jan 20, 2010, at 03:29 PM, David Lyon wrote:

>> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>>
>> I'd be surprised if any of the big 3 DVCS developers would actually /want/
>> their stuff in the stdlib.
>
>If they ask, they'll get told they're motorbike-shedding. "It's better
>if their users ask". So here I am as a user doing things the 'right'
>way.

Actually, you're not.  It's not up to the Python community to initiate this.
If you really want this, you should engage with the relevant DVCS communities
and push them to request it.

>Side benefits are that it can update everything else along
>with it at the same time. User Apps, Packages, whatever.

I get that.  Heck, I still run one Gentoo server which I think is as close to
the edge you're describing as I'm comfortable with.  It's all great until the
wheels come off and then it can take *days* to get a functioning system
again.

The big difference is that I rely on my DVCS to keep one small thing, or a few
variants of the same thing, all sane.  But I rely on my distribution vendor to
keep a thousand complex, interdependent, interacting, sometimes conflicting
things sane and working.

>Point : The fact that SCMs are two way is great in
>        a production environment. No packaging solution
>        can come close.

Try talking with some hard-core operations guys, the folks with the keys to
the data centers, who work tireless, insanely hours keeping incredibly complex
systems running with very little downtime.  I think you'd get a different
perspective to put it mildly. :)

to-sleep-perchance-to-dream-ly y'rs,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/0def1cc9/attachment-0004.pgp>

From david.lyon at pythontest.org  Wed Jan 20 06:10:15 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 16:10:15 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <87wrzdfm1y.fsf@benfinney.id.au>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
Message-ID: <1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>

> "David Lyon" <david.lyon at pythontest.org> writes:
>
>> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
>> and BZR can't make it into the stdlib, then there is no hope for even
>> newer code to get in there.
>
> Those are applications, not libraries. Applications don't belong in the
> standard library.

Haha funny..

Well using that logic, distutils is an application..

Are you saying that distutils should be removed? That
is most certainly an application.

Lets not get too pedantic here. Mercurial and bzr have a built
in API that can be called in a library like way. It's true they
also have a command line interface in the same way that distutils
does.

I'm not saying anything negative about distutils. Given that
Tarek has an upcoming Pycon presentation where the program
talks about a distutils revamp.

I'm hoping that he can find some young 20 yr olds and
put a cool web interface on that thing. Given that there
are empty sprints at pycon. It couldn't hurt to throw
that challenge out. Anyway, we'll see..


David




From ben+python at benfinney.id.au  Wed Jan 20 06:32:10 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:32:10 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
	<20100119235036.57f6e867@freewill>
Message-ID: <87iqaxfijp.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> On Jan 20, 2010, at 03:29 PM, David Lyon wrote:
> >So here I am as a user doing things the 'right' way.
>
> Actually, you're not. It's not up to the Python community to initiate
> this. If you really want this, you should engage with the relevant
> DVCS communities and push them to request it.

Where ?push? must be strictly limited by a continual awareness that the
whole idea could just be bad.

If you find yourself in a tiny minority pushing for a change, it *could*
be that you have a great idea and the vast majority don't realise it
yet. But you must be realistic about the likelihood that the change is a
very *bad* idea, and frequently evaluate it for signs of that.

-- 
 \     ?I used to think that the brain was the most wonderful organ in |
  `\   my body. Then I realized who was telling me this.? ?Emo Philips |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Wed Jan 20 06:34:07 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:34:07 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
	<1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87eillfigg.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Well using that logic, distutils is an application..

Distutils is an application, the function of which is essential to
allowing sane development of Python packages. It's a special case. We
need to strictly limit the number of special cases, not gleefully add to
them.

-- 
 \        ?I'd take the awe of understanding over the awe of ignorance |
  `\                                          any day.? ?Douglas Adams |
_o__)                                                                  |
Ben Finney



From matthieu.brucher at gmail.com  Wed Jan 20 07:49:36 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 07:49:36 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>

> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

I think there are several points that make them not includable in Python:
- git is not written in Python
- bzr and mercurial have a life cycle much shorter than Python's, it's
the same issue than with other libraries where another community
develops them.

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From david.lyon at pythontest.org  Wed Jan 20 08:20:02 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 18:20:02 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
Message-ID: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>


Matthieu,

>> I'd be happy to see bzr and mercurial and git all made it together
>> into the stdlib for python 3. That would give a superb updating
>> mechanism for python that would propel python well beyond
>> the dinosaur badlands of CPAN and other languages.
>
> I think there are several points that make them not includable in Python:
> - git is not written in Python
> - bzr and mercurial have a life cycle much shorter than Python's, it's
> the same issue than with other libraries where another community
> develops them.

That's only two points. :-)

On 1; If that's true, I won't mention git again.

On 2; Who knows what their life cycle is. CVS is pretty much
      dead, and svn looks like it is on the way out.
      I can't think of how anything could be better than
      mercurial or bzr but I know I will be proved wrong.

At the end of the day, we are making a decision about whether
the language is 'set-in-stone' or whether it is still
evolving.

To me, Python 1.x had it's own distinct "era", as has
Python 2.x

Hoping that the Python 3 era can be a little more flexible
and perphaps "cleaner" than the 2.x era is all that I am
thinking here.

Have a nice day

David





From matthieu.brucher at gmail.com  Wed Jan 20 09:05:19 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 09:05:19 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001200005j6344672fo99826cc2e8648141@mail.gmail.com>

> That's only two points. :-)

In French, we say that several starts with 2 ;)

> On 1; If that's true, I won't mention git again.

I tis, you can check on the git repository (it's a mix of C, perl,
shell scripts, Python, ...)

> On 2; Who knows what their life cycle is.

You can check on their websites, their cycles are far shorter than
Python minor releases (several months vs several years).

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From stephen at xemacs.org  Wed Jan 20 11:22:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 20 Jan 2010 19:22:55 +0900
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <87aaw9gjnk.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > (Besides, git in the stdlib doesn't make much sense :).

"Dulwich."


From solipsis at pitrou.net  Wed Jan 20 12:28:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Jan 2010 11:28:16 +0000 (UTC)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100120T122742-162@post.gmane.org>

David Lyon <david.lyon <at> pythontest.org> writes:
> 
> I think I'll have better chances with PEPs.
> 
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
> [snip]

This is python-ideas material, can you take it there? Thank you.

Antoine.




From ncoghlan at gmail.com  Wed Jan 20 13:16:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 22:16:34 +1000
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>	<20100119185439.299660c7@freewill>	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B56F422.5010607@gmail.com>

David Lyon wrote:
> On 2; Who knows what their life cycle is. CVS is pretty much
>       dead, and svn looks like it is on the way out.
>       I can't think of how anything could be better than
>       mercurial or bzr but I know I will be proved wrong.

I believe you misunderstood what Matthieu meant by life cycle there:
think "release cycle". If a project pushes out new releases
significantly more often than every 18-24 months (as is currently true
for all of the major SCM tools), then that fact alone makes it a very
bad fit for the Python standard library.

And centralised source control will be going strong for years. The DVCS
approach may be great for the open source world, but the gains are far
more limited in a closed source shop (especially a group writing
internal corporate applications which doesn't need to keep many, if any,
maintenance branches going).

If we weren't dealing with 4 active branches, the DVCS discussion would
have got a lot less traction with the core developers - aside from
better handling of multiple lines of development, most of the benefits
of the switch to a DVCS accrue to people without commit access to the
SVN repository.

Anyway, we've wandered far afield from legit python-dev topics now. Any
further ideas about super_mega_easy_install functionality that can pull
code from source control systems and build it rather than requiring
prebuild source tarballs should be directed to python-ideas (they
probably need to bake more even before they make an appearance on
distutils-sig).

Cheers,
Nick.

P.S. As Jesse said... your enthusiasm is great, but please don't assume
that some inherent conservatism on the part of other developers is
automatically evil or the result of a failure to see your point. A lot
of people around the world rely on our stuff every day. We owe it to
them to be measured in our actions and to put serious thought into any
major changes or additions we make to the language and the standard
library. For the current stage of its development, Python 3 is in a good
place from our point of view - its major carrot has really always been
the better Unicode support it offers, and the ever-increasing
globalisation of the web will create more and more pressure pushing
developers in that direction as the years go by. Sure, Python 3 cleans
up assorted other things as well, but the change to the text processing
model is the big one that is fundamentally incompatible with the
architecture of the 2.x series. Compared to that change, everything else
is just tinkering.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ziade.tarek at gmail.com  Fri Jan  1 16:07:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:07:20 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04D81D51@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<94bdd2610912141458m52da21ear4970506a37214e6@mail.gmail.com>
	<4dab5f760912230700l38a597f7l855bb2304025d6d5@mail.gmail.com>
	<F2594678-D242-410D-975F-AABEE2EBB87B@twistedmatrix.com>
Message-ID: <94bdd2611001010707h4043ff64x7476049e435852b@mail.gmail.com>

2009/12/23 Glyph Lefkowitz <glyph at twistedmatrix.com>:
[..]
> 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages".
>
> The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. ?In other words it's got "bin", "lib", "share", "etc", etc.. ?~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation.

Right, good idea

Tarek

-- 
Tarek Ziad? | http://ziade.org


From ziade.tarek at gmail.com  Fri Jan  1 16:32:27 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 1 Jan 2010 16:32:27 +0100
Subject: [Python-Dev] First draft of "sysconfig"
In-Reply-To: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
References: <94bdd2610912121202l48d39325q6f4cdcd73f972d5c@mail.gmail.com>
Message-ID: <94bdd2611001010732o38a563bcu6d0cc9122ed5c88f@mail.gmail.com>

On Sat, Dec 12, 2009 at 9:02 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello,
>
> A while ago I've proposed to refactor the APIs that provides access to
> the installation paths and configuration variables at runtime into a
> single module called "sysconfig", and make it easier for all
> implementations to work with them.

I've applied the suggestions made in this thread.

If no one objects, here's what I am going to do now:

I'll merge this change in the trunk, (I'll drop the branch changesets
in favor of one single, clean changeset) and start to work on the
corresponding doc, so more people will be able to give some feedback
on the APIs once the second 2.7 alpha is out.

Then, in a second step, I'll work on the removal of the Makefile and
pyconfig.h dependencies by adding a _synconfig.py file as suggested
earlier, that will be created during the build process.

Regards,
Tarek


From status at bugs.python.org  Fri Jan  1 18:07:11 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Jan 2010 18:07:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100101170711.A5AAF78654@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/25/09 - 01/01/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2537 open (+22) / 16902 closed (+19) / 19439 total (+41)

Open issues with patches:  1016

Average duration of open issues: 705 days.
Median duration of open issues: 462 days.

Open Issues Breakdown
   open  2504 (+22)
pending    32 ( +0)

Issues Created Or Reopened (42)
_______________________________

doc: Code is not always colored                                  12/30/09
CLOSED http://bugs.python.org/issue7487    reopened flox                          
                                                                               

documention buglet for PyBuffer                                  12/26/09
CLOSED http://bugs.python.org/issue7577    created  ronaldoussoren                
       easy                                                                    

Behavior of operations on a closed file object is not documented 12/26/09
       http://bugs.python.org/issue7578    created  blep                          
                                                                               

Patch to add docstrings to msvcrt                                12/26/09
       http://bugs.python.org/issue7579    created  brian.curtin                  
       patch                                                                   

distutils makefile from python.org installer doesn't work on OS  12/27/09
       http://bugs.python.org/issue7580    created  bskaplan                      
                                                                               

incomplete doc of zlib                                           12/27/09
CLOSED http://bugs.python.org/issue7581    created  coolwanglu                    
                                                                               

[patch] diff.py to use iso timestamp                             12/27/09
       http://bugs.python.org/issue7582    created  techtonik                     
       patch                                                                   

doctest should normalize tabs when comparing output              12/27/09
       http://bugs.python.org/issue7583    created  techtonik                     
                                                                               

datetime.rfcformat() for Date and Time on the Internet           12/27/09
       http://bugs.python.org/issue7584    created  techtonik                     
                                                                               

[patch] difflib should separate filename from timestamp with tab 12/27/09
       http://bugs.python.org/issue7585    created  techtonik                     
       patch                                                                   

Typo in collections documentation                                12/28/09
CLOSED http://bugs.python.org/issue7586    created  nneonneo                      
                                                                               

Python 3.1.1 source build issues                                 12/28/09
CLOSED http://bugs.python.org/issue7587    created  sharmaag                      
                                                                               

unittest.TestCase.shortDescription isn't short anymore           12/28/09
       http://bugs.python.org/issue7588    created  exarkun                       
                                                                               

setup.py shouldn't try to build nis module when nis headers aren 12/28/09
CLOSED http://bugs.python.org/issue7589    created  Arfrever                      
       patch                                                                   

'exceptions' module mentioned in documentation                   12/28/09
CLOSED http://bugs.python.org/issue7590    created  July                          
                                                                               

test_get_platform fails on 3.1                                   12/28/09
       http://bugs.python.org/issue7591    created  pitrou                        
       buildbot                                                                

ssl module documentation: SSLSocket.unwrap description shown twi 12/28/09
       http://bugs.python.org/issue7592    created  mnewman                       
                                                                               

Computed-goto patch for RE engine                                12/29/09
       http://bugs.python.org/issue7593    created  akuchling                     
       patch, needs review                                                     

shlex refactoring                                                12/29/09
       http://bugs.python.org/issue7594    created  ferringb                      
       patch, needs review                                                     

Doc typo for select.kevent()                                     12/29/09
CLOSED http://bugs.python.org/issue7595    created  whit537                       
                                                                               

test_logging sometimes fails                                     12/29/09
CLOSED http://bugs.python.org/issue7596    created  pitrou                        
                                                                               

curses.use_env implementation error                              12/30/09
       http://bugs.python.org/issue7597    created  kanru                         
       patch                                                                   

Cannot install, problems with assembly?                          12/30/09
CLOSED http://bugs.python.org/issue7598    created  sh.00                         
                                                                               

(c)ElementTree can produce invalid XML                           12/30/09
       http://bugs.python.org/issue7599    created  jwilk                         
                                                                               

interpreter crashes before start                                 12/30/09
CLOSED http://bugs.python.org/issue7600    created  mhh91                         
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc         12/30/09
CLOSED http://bugs.python.org/issue7601    created  sadhak                        
                                                                               

Doc: make clean and make update do not delete or update Doc/tool 12/30/09
CLOSED http://bugs.python.org/issue7602    created  flox                          
                                                                               

There is no 'seq' type                                           12/30/09
CLOSED http://bugs.python.org/issue7603    created  tom66                         
                                                                               

delattr __slots__ inconsistancy                                  12/30/09
CLOSED http://bugs.python.org/issue7604    created  ferringb                      
                                                                               

test_cmd_line fails with non-ascii path                          12/30/09
       http://bugs.python.org/issue7605    created  pitrou                        
                                                                               

test_xmlrpc fails with non-ascii path                            12/30/09
       http://bugs.python.org/issue7606    created  pitrou                        
                                                                               

stringlib fastsearch could be improved on 64-bit builds          12/30/09
       http://bugs.python.org/issue7607    created  pitrou                        
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.             12/30/09
CLOSED http://bugs.python.org/issue7608    created  alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                   12/31/09
CLOSED http://bugs.python.org/issue7609    created  Arfrever                      
       patch                                                                   

Cannot use both read and readline method in same ZipExtFile obje 12/31/09
       http://bugs.python.org/issue7610    created  lucifer                       
                                                                               

shlex not posix compliant when parsing "foo#bar"                 12/31/09
       http://bugs.python.org/issue7611    created  jjdmol2                       
                                                                               

Fix "pass and object" typo in Library Reference / Built-in Types 12/31/09
CLOSED http://bugs.python.org/issue7612    created  ygale                         
       patch                                                                   

[cppcheck] found a regression : Invalid number of character ((). 12/31/09
CLOSED http://bugs.python.org/issue7613    created  ettl.martin                   
       patch                                                                   

Python 2.6.4 segfaults                                           12/31/09
CLOSED http://bugs.python.org/issue7614    created  ttsiod                        
                                                                               

unicode_escape codec does not escape quotes                      01/01/10
       http://bugs.python.org/issue7615    created  rhansen                       
       patch                                                                   

test_memoryview test_setitem_writable failures with Intel ICC    01/01/10
       http://bugs.python.org/issue7616    created  ivank                         
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option 01/01/10
       http://bugs.python.org/issue7617    created  Arfrever                      
       patch                                                                   



Issues Now Closed (46)
______________________

True division of integers could be more accurate                  715 days
       http://bugs.python.org/issue1811    mark.dickinson                
       patch                                                                   

API to clear most free lists                                      690 days
       http://bugs.python.org/issue2019    amaury.forgeotdarc            
       patch                                                                   

unit test UnicodeWarning                                          687 days
       http://bugs.python.org/issue2100    ezio.melotti                  
                                                                               

test_disutils fails                                               568 days
       http://bugs.python.org/issue3054    ronaldoussoren                
       patch                                                                   

test_formatdate_usegmt fails on non-englisch locale               451 days
       http://bugs.python.org/issue4046    r.david.murray                
                                                                               

"??" in u"foo" raises a misleading error                          410 days
       http://bugs.python.org/issue4328    r.david.murray                
                                                                               

zipfile - add __exit__ attribute to make ZipFile object compatib  287 days
       http://bugs.python.org/issue5511    ezio.melotti                  
       patch                                                                   

test_urllib2 fails - urlopen error file not on local host         271 days
       http://bugs.python.org/issue5625    orsenthil                     
                                                                               

Improve --with-dbmliborder option                                 170 days
       http://bugs.python.org/issue6491    benjamin.peterson             
       patch                                                                   

Move the special-case for integer objects out of PyBytes_FromObj  141 days
       http://bugs.python.org/issue6687    alexandre.vassalotti          
       patch, 26backport                                                       

setup.py fails to find headers of system libffi                   104 days
       http://bugs.python.org/issue6943    benjamin.peterson             
       patch, easy                                                             

The replacement suggested for callable(x) in py3k is not	equival   92 days
       http://bugs.python.org/issue7006    benjamin.peterson             
       patch                                                                   

C/API - Document exceptions                                        89 days
       http://bugs.python.org/issue7033    lekma                         
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi   35 days
       http://bugs.python.org/issue7381    georg.brandl                  
       patch                                                                   

optparse Documentation References Example Files that do not Exis   30 days
       http://bugs.python.org/issue7404    georg.brandl                  
       patch                                                                   

datetime.datetime.isoformat truncation problem                     29 days
       http://bugs.python.org/issue7413    amaury.forgeotdarc            
       patch, needs review                                                     

doc: Code is not always colored                                     0 days
       http://bugs.python.org/issue7487    georg.brandl                  
                                                                               

float('nan')**2 != nan. float('nan')**2 error 33 on windows        13 days
       http://bugs.python.org/issue7534    mark.dickinson                
       patch                                                                   

If a generator raises TypeError when being unpacked, an unrelate   10 days
       http://bugs.python.org/issue7548    amaury.forgeotdarc            
                                                                               

ctypes doc improvement: c_char_p                                    6 days
       http://bugs.python.org/issue7569    georg.brandl                  
                                                                               

Strange issue : cursor.commit() with sqlite                         5 days
       http://bugs.python.org/issue7572    ghaering                      
                                                                               

tes_math fails Mac OS X 10.4 due to OverflowError in test_mtestf    5 days
       http://bugs.python.org/issue7575    mark.dickinson                
       patch                                                                   

documention buglet for PyBuffer                                     2 days
       http://bugs.python.org/issue7577    georg.brandl                  
       easy                                                                    

incomplete doc of zlib                                              0 days
       http://bugs.python.org/issue7581    amaury.forgeotdarc            
                                                                               

Typo in collections documentation                                   0 days
       http://bugs.python.org/issue7586    georg.brandl                  
                                                                               

Python 3.1.1 source build issues                                    0 days
       http://bugs.python.org/issue7587    amaury.forgeotdarc            
                                                                               

setup.py shouldn't try to build nis module when nis headers aren    1 days
       http://bugs.python.org/issue7589    benjamin.peterson             
       patch                                                                   

'exceptions' module mentioned in documentation                      1 days
       http://bugs.python.org/issue7590    georg.brandl                  
                                                                               

Doc typo for select.kevent()                                        0 days
       http://bugs.python.org/issue7595    georg.brandl                  
                                                                               

test_logging sometimes fails                                        1 days
       http://bugs.python.org/issue7596    ezio.melotti                  
                                                                               

Cannot install, problems with assembly?                             0 days
       http://bugs.python.org/issue7598    ezio.melotti                  
                                                                               

interpreter crashes before start                                    0 days
       http://bugs.python.org/issue7600    r.david.murray                
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            1 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

Doc: make clean and make update do not delete or update Doc/tool    0 days
       http://bugs.python.org/issue7602    georg.brandl                  
                                                                               

There is no 'seq' type                                              0 days
       http://bugs.python.org/issue7603    benjamin.peterson             
                                                                               

delattr __slots__ inconsistancy                                     0 days
       http://bugs.python.org/issue7604    benjamin.peterson             
                                                                               

PyUnicode_FromFormatV handles %R and %S incorrectly.                0 days
       http://bugs.python.org/issue7608    alexandre.vassalotti          
                                                                               

Add --with-system-expat option                                      1 days
       http://bugs.python.org/issue7609    benjamin.peterson             
       patch                                                                   

Fix "pass and object" typo in Library Reference / Built-in Types    0 days
       http://bugs.python.org/issue7612    ezio.melotti                  
       patch                                                                   

[cppcheck] found a regression : Invalid number of character (().    0 days
       http://bugs.python.org/issue7613    ezio.melotti                  
       patch                                                                   

Python 2.6.4 segfaults                                              0 days
       http://bugs.python.org/issue7614    mark.dickinson                
                                                                               

Fwd: Debian Bug#42318: urllib.py has problems with malformed pro 3436 days
       http://bugs.python.org/issue210849  shinnosuke                    
                                                                               

urllib / urllib2 should cache 301 redirections                   2425 days
       http://bugs.python.org/issue735515  pitrou                        
                                                                               

fast modular exponentiation                                      2084 days
       http://bugs.python.org/issue936813  mark.dickinson                
       patch                                                                   

bdist_deb - Debian packager                                       316 days
       http://bugs.python.org/issue1054967 astraw                        
       patch                                                                   

Carbon.Scrap.PutScrapFlavor                                       987 days
       http://bugs.python.org/issue1700507 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 11 Add Option to Bind to a Local IP Address in httplib.py           462 days
open    http://bugs.python.org/issue3972   

  8 fast modular exponentiation                                     2084 days
closed  http://bugs.python.org/issue936813 

  6 [patch] difflib should separate filename from timestamp with ta    5 days
open    http://bugs.python.org/issue7585   

  6 [patch] diff.py to use iso timestamp                               5 days
open    http://bugs.python.org/issue7582   

  6 Implement fastsearch algorithm for rfind/rindex                   24 days
open    http://bugs.python.org/issue7462   

  5 test_xmlrpc fails with non-ascii path                              2 days
open    http://bugs.python.org/issue7606   

  5 test_logging sometimes fails                                       1 days
closed  http://bugs.python.org/issue7596   

  5 Patch to add docstrings to msvcrt                                  6 days
open    http://bugs.python.org/issue7579   

  5 Distutils-based installer does not detect 64bit versions of Pyt  126 days
open    http://bugs.python.org/issue6792   

  5 _sha256 et al. encode to UTF-8 by default                         17 days
open    http://bugs.python.org/issue3745   




From stefan_ml at behnel.de  Sun Jan  3 09:11:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Jan 2010 09:11:16 +0100
Subject: [Python-Dev] Providing support files to assist 3.x extension
	authors
In-Reply-To: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
References: <99e0b9530912191638u757d2ce5mdf15394482cc19c0@mail.gmail.com>
Message-ID: <hhpjf3$lk1$1@ger.gmane.org>

Case Vanhorsen, 20.12.2009 01:38:
> When I ported gmpy (Python to GMP multiple precision library) to
> Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I
> found the code to slightly faster and cleaner than using PyLong_AsLong
> and checking for overflow.

You might want to look at the code Cython generates for integer type 
conversions. We use specialised coercion code that gets generated 
on-the-fly to convert Python long/int from and to all sorts of C integer 
types with compile time (portability) and runtime size/value checks. 
Depending on your needs, this may or may not be faster than the above C-API 
function.

Stefan



From david.lyon at preisshare.net  Mon Jan  4 00:42:15 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 10:42:15 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
Message-ID: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>


> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>
> This new operator removes the ambiguity the original proposal had,
> without making it more
> complex for common use cases. So if you dislike it, you will need to
> propose something
> else that also fixes the ambiguity we had.

Ok.

> Environment markers
>..
>Here are some example of fields using such markers:
>
>Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'

  Requires-Dist: [Windows] pywin32 1.0+

That's simpler, shorter, and less ambiguous. Easier to
parse for package managers.

David





From python at mrabarnett.plus.com  Mon Jan  4 01:14:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 04 Jan 2010 00:14:43 +0000
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4132F3.7020905@mrabarnett.plus.com>

David Lyon wrote:
>> On Mon, Dec 28, 2009 at 1:15 AM,  Tarek Ziade wrote:
>>
>> This new operator removes the ambiguity the original proposal had,
>> without making it more
>> complex for common use cases. So if you dislike it, you will need to
>> propose something
>> else that also fixes the ambiguity we had.
> 
> Ok.
> 
>> Environment markers
>> ..
>> Here are some example of fields using such markers:
>>
>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.
> 
'win32' is more specific than 'Windows' and, to me, '1.0+' means
'>=1.0', not '>1.0'.


From martin at v.loewis.de  Mon Jan  4 01:21:53 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 04 Jan 2010 01:21:53 +0100
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4134A1.5090203@v.loewis.de>

>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
> 
>   Requires-Dist: [Windows] pywin32 1.0+
> 
> That's simpler, shorter, and less ambiguous. Easier to
> parse for package managers.

Don't you want the PEP to complete? Why this bike-shedding?

I can agree it's shorter. I can't agree that it's simpler,
or less ambiguous.

Regards,
Martin


From ssteinerx at gmail.com  Mon Jan  4 01:29:14 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sun, 3 Jan 2010 19:29:14 -0500
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
	Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>	<75d5dd69b850e9bd793b43618327e655@preisshare.net>	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>	<4B3333DF.40705@v.loewis.de>	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>	<4B334109.2060708@v.loewis.de>
	<4B346ACE.2030400@gmail.com>	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <DF433474-9F8E-40BA-B0B1-D3021CAC6317@gmail.com>


On Jan 3, 2010, at 7:21 PM, Martin v. L?wis wrote:

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>> 
>>  Requires-Dist: [Windows] pywin32 1.0+
>> 
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
> 
> Don't you want the PEP to complete? Why this bike-shedding?

Really....

Enough, already!

S



From david.lyon at preisshare.net  Mon Jan  4 01:47:56 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 4 Jan 2010 11:47:56 +1100 (EST)
Subject: [Python-Dev] Proposing PEP 345 : Metadata for Python Software
 Packages 1.2
In-Reply-To: <4B4134A1.5090203@v.loewis.de>
References: <94bdd2610912211625k6a52dc19ue7dd7cad1e4f655c@mail.gmail.com>
	<75d5dd69b850e9bd793b43618327e655@preisshare.net>
	<871vilqhsy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<e626d66af534e03f22749f3dd4a6ff73@preisshare.net>
	<4B3333DF.40705@v.loewis.de>
	<94bdd2610912240152r6131ec9ay2ada83f66aa17b35@mail.gmail.com>
	<4B334109.2060708@v.loewis.de> <4B346ACE.2030400@gmail.com>
	<94bdd2610912271601h69b081adsc871c849d1e2e1cc@mail.gmail.com>
	<4203.115.128.50.49.1261959349.squirrel@syd-srv02.ezyreg.com>
	<94bdd2610912271637m7b3df609m780622f74ee5daa@mail.gmail.com>
	<1255.218.214.45.58.1262562135.squirrel@syd-srv02.ezyreg.com>
	<4B4134A1.5090203@v.loewis.de>
Message-ID: <1349.218.214.45.58.1262566076.squirrel@syd-srv02.ezyreg.com>


Hi Martin, Happy New Year,

>>> Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
>>
>>   Requires-Dist: [Windows] pywin32 1.0+
>>
>> That's simpler, shorter, and less ambiguous. Easier to
>> parse for package managers.
>
> Don't you want the PEP to complete? Why this bike-shedding?

Well, I'm just helping out by pointing out some simpler ways
as Tarek asked me. I was only answering his question. I was out
celebrating so it took longer to reply than normal.

Bike-Shedding ? Me ? which bikeshed? wanting simple?

Anyway, I'm just reading the PEPs and commenting. If there
are some alterations that can be done, lets discuss them.

> I can agree it's shorter.
> ..

Cool.

What I'd really like is a 'Code-Repository:' keyword
so that we can install programs/libraries directly into
a system.

I feel that this would really simplify the packaging
landscape, making it much easier for the scientific
community and others to do python software installs.

This would allow us to perphaps have something even
*more modern* than CPAN.

So yes, getting PEP-345 right is important to me.

Have a nice day.

David





From abpillai at gmail.com  Mon Jan  4 10:08:05 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Mon, 4 Jan 2010 14:38:05 +0530
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <8548c5f31001040108v635a78ech33521371c6c50e82@mail.gmail.com>

On Sun, Dec 27, 2009 at 11:21 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Tue, Dec 15, 2009 at 12:37 AM, Steven Bethard
> <steven.bethard at gmail.com> wrote:
> >
> > If you're only concerned about 2.X, then yes, optparse will *never* be
> > removed from 2.X. There will be a deprecation note in the 2.X
> > documentation but deprecation warnings will only be issued when the -3
> > flag is specified. Please see the "Deprecation of optparse" section of
> > the PEP:
> >
> > http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
>
 Ref http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse .

 Considering that optparse will be deprecated like 5 years from now.
 I think this point is moot. The deprecation strategy IMHO is
 perhaps way too conservative. Maybe it should be deprecated
 faster ;)




-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100104/e0cabd8c/attachment-0005.html>

From fetchinson at googlemail.com  Mon Jan  4 10:32:10 2010
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Mon, 4 Jan 2010 10:32:10 +0100
Subject: [Python-Dev] Pronouncement on PEP 389: argparse?
In-Reply-To: <d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
References: <d11dcfba0912141004u6728deb1nc596c5afd1480e27@mail.gmail.com>
	<b654cd2e0912141022n1a9afc7fr19bf243ba7b11359@mail.gmail.com>
	<d11dcfba0912141043r1e63a397g20374bc927d7f135@mail.gmail.com>
	<24ea26600912141112i31a9eba7j7d06837456508094@mail.gmail.com>
	<d11dcfba0912141200k1045d1b0w322de2753ab35ca4@mail.gmail.com>
	<4B26A41F.5020009@gmail.com>
	<24ea26600912141310w77d07c42hc244f9ecc1642b9f@mail.gmail.com>
	<d11dcfba0912141437h66ee8fa8he8c9c2287b7dbdd3@mail.gmail.com>
	<d34314100912270951x4fd96e67mde5ec0a72b26374f@mail.gmail.com>
Message-ID: <fbe2e2101001040132o22302d2ct72b705ec046bcc46@mail.gmail.com>

>> If you're only concerned about 2.X, then yes, optparse will *never* be
>> removed from 2.X. There will be a deprecation note in the 2.X
>> documentation but deprecation warnings will only be issued when the -3
>> flag is specified. Please see the "Deprecation of optparse" section of
>> the PEP:
>>
>> http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse
>
> I do not think that optparse should be deprecated at. It is good at
> what it does and its limitations make its starting point less
> confusing for people with different backgrounds that Python.
>
> Compare:
> http://docs.python.org/library/optparse.html
> http://argparse.googlecode.com/svn/tags/r101/doc/index.html
>
> argparse should be recommended as advanced and more featured variant
> of optparse - that goes without saying, but forcing people to switch
> and annoying them with deprecation messages brings only headache.

As has been noted already nobody is forcing people to switch. Optparse
will be available as a separate package and everybody will be free to
install it and will not have any deprecation messages anywhere.

> Just
> like optparse is better getopt, the latter could also be useful for
> people coming from other languages and porting their libraries to
> Python.
>
> I would better concentrate on real code examples how argparse solves
> problems and would really want to see
> http://www.python.org/dev/peps/pep-0389/#out-of-scope-various-feature-requests
> implemented until argparse enters phase where backward incompatible
> changes in API are impossible.
>
> I would prefer to see PEP 389 as a document describing proposed
> solutions to argument parsing problems rather than petition to replace
> one library with another. So, it should display common argument
> parsing scenarios (use cases) with examples that are also useful
> recipes for documentation. I guess more than 90% people here doesn't
> have time to read argparse methods descriptions to see what they could
> be useful for.



-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From olemis at gmail.com  Mon Jan  4 15:24:05 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 4 Jan 2010 09:24:05 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
Message-ID: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>

On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
> Thanks for the quick response.
>
> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> When I made that change, I didn't know that the __unittest "hack" was
>> being used elsewhere outside of unittest, so I felt fine replacing it
>> with another. While I still consider it an implementation detail, I
>> would be ok with exposing an "official" API for this. Perhaps
>> __unittest_ignore_traceback?
>
> Well, bazaar has had the trick for a couple of years, and googling
> around now turns up some other projects using it or thinking about it:
> <http://github.com/gfxmonk/mocktest/commit/b5e94f7ee06ab627cea2c9cf90d0aeb63ee5a698>
> <http://bitbucket.org/uche/amara/changeset/eeaf69f48271/>
> <http://twistedmatrix.com/trac/ticket/4127>
>

Add `dutest` and probably `nose` [2]_ and ...

> Reinstating the old implementation (with the same name) would mean
> that existing code would keep working with Python 2.7

IOW ... if it is removed all the libraries will have to change and
check for the Py version and ... (probably not a big deal depending on
the details of the ?solution?)

> but maybe a
> discussion could start about a new, less hacky, way of doing the same
>

I am strongly -1 for modifying the classes in ?traditional? unittest
module [2]_ (except that I am strongly +1 for the package structure
WITHOUT TOUCHING anything else ...) , and the more I think about it I
am more convinced ... but anyway, this not a big deal (and in the end
what I think is not that relevant either ... :o). So ...

pass

> May not be worthwhile making life more complicated though,
> there aren't *that* many unittest-extending projects.
>

But every library extending `unittest` will do it or otherwise
not-so-useful error messages will be displayed
;o)

PS: Happy New Year ! BTW

.. [1] [feature] nosexml should omit trace frames where `__unittest...
        (http://code.google.com/p/python-nosexml/issues/detail?id=4#c0)

.. [2] Assessment of unittest 2.7 API : new features and opinions
        (http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Free new ripit 1.3.3 Download - mac software  -
http://feedproxy.google.com/~r/TracGViz-full/~3/KFwyUTKH0vI/


From juanfhj at gmail.com  Tue Jan  5 17:10:16 2010
From: juanfhj at gmail.com (Juan Fernando Herrera J.)
Date: Tue, 5 Jan 2010 11:10:16 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards compatibility
Message-ID: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>

How about a new python 3 release with (possibly partial) backwards
compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
software hasn't been ported to it. I'm eager to use 3, but paradoxically,
the 3 release makes me rather stuck with 2.6. Excuse me if this has been
suggested in the past.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/7839cf7b/attachment-0007.htm>

From fuzzyman at voidspace.org.uk  Tue Jan  5 17:50:15 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 05 Jan 2010 16:50:15 +0000
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <4B436DC7.8040605@voidspace.org.uk>

On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
> How about a new python 3 release with (possibly partial) backwards 
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way 
> major software hasn't been ported to it. I'm eager to use 3, but 
> paradoxically, the 3 release makes me rather stuck with 2.6. Excuse me 
> if this has been suggested in the past.
>    

I don't know about other developers, but I certainly expected Python 3 
adoption to take a few years due to libraries only gradually making the 
jump. I also expected there to be nervousness and pain around the 
migration as well.

I think in fact that libraries *are* migrating and there are lots of 
encouraging signs. As a community we need to do all we can to promote 
Python 3 adoption, but I don't think there is much reason to be worried 
(or complacent for that matter).

All the best,

Michael Foord

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/d7b6baac/attachment-0007.htm>

From brian.curtin at gmail.com  Tue Jan  5 18:21:31 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 5 Jan 2010 11:21:31 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>

On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:

> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
The proper route to take, in my opinion, is to see what 2.x libraries you
are using that are not 3.x compatible, run 2to3 on them, then run their test
suite, and see where you get. Submit a patch or two to the library and see
what happens -- it at least gets the wheels in motion.

I'm sure everyone out there would like to dive into supporting 3.x, but it's
tough to prioritize when you are up to your eyeballs with 2.x bugs in your
tracker. Look at Twisted (
http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3)
-- over 1000 issues, ~5 developers -- 3.x support won't be here tomorrow,
but it's on the horizon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/67a4e6e2/attachment-0007.htm>

From guido at python.org  Tue Jan  5 18:23:08 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 5 Jan 2010 09:23:08 -0800
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B436DC7.8040605@voidspace.org.uk>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<4B436DC7.8040605@voidspace.org.uk>
Message-ID: <ca471dc21001050923i2ca37f6ej543faf0981c43b13@mail.gmail.com>

On Tue, Jan 5, 2010 at 8:50 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 05/01/2010 16:10, Juan Fernando Herrera J. wrote:
>
> How about a new python 3 release with (possibly partial) backwards
> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.
>
>
> I don't know about other developers, but I certainly expected Python 3
> adoption to take a few years due to libraries only gradually making the
> jump. I also expected there to be nervousness and pain around the migration
> as well.
>
> I think in fact that libraries *are* migrating and there are lots of
> encouraging signs. As a community we need to do all we can to promote Python
> 3 adoption, but I don't think there is much reason to be worried (or
> complacent for that matter).
>

Michael is right, but doesn't answer the OP's implied question "why can't
3.x be made backwards compatible with 2.6?"

Unfortunately the answer isn't easy. I wish it was "because we didn't have
time to do it" (since then the solution would be simply to call for more
volunteers to help out) -- but unfortunately, it comes closer to "we have no
idea how to do it."

Py3k was designed to be backwards incompatible, because that gives the most
long-term benefit of the language improvements. (Perhaps a better way of
saying this is that in the design of language improvements,
feature-by-feature backwards compatibility was intentionally not a
requirement, even though keeping most of the language mostly unchanged *was*
a requirement.)

In principle it would be possible to be more backwards compatible at the
syntactic level, but that's not where the pain of porting code lies -- the
major hurdles are typically he new way of thinking about Unicode (bytes vs.
text strings, instead of 8-bit-strings vs. Unicode strings), and the changed
semantics of dictionary keys and related APIs. Since these issues typically
exist across modules (passing strings and dicts between modules is common),
recognizing individual modules as "2.x" vs. "3.x" isn't possible.

Note, by the way, that you won't be "stuck" on 2.6 forever -- 2.7 alpha 1
was already released.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/eaba2f9b/attachment-0007.htm>

From regebro at gmail.com  Tue Jan  5 18:52:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 18:52:20 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
Message-ID: <319e029f1001050952o71cb29afh66445550beeb99c2@mail.gmail.com>

On Tue, Jan 5, 2010 at 17:10, Juan Fernando Herrera J.
<juanfhj at gmail.com> wrote:
> I'm eager to use 3, but paradoxically,
> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
> suggested in the past.

Yes. Python 3 is not what you want to use today if you write
applications. If you write libraries 2010 is the year to port, IMO.
With some luck, 2011 will be the year to start porting applications
properly. We'll see.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ianb at colorstudy.com  Tue Jan  5 20:00:49 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 13:00:49 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
Message-ID: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>

On Tue, Jan 5, 2010 at 11:21 AM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Tue, Jan 5, 2010 at 10:10, Juan Fernando Herrera J. <juanfhj at gmail.com>wrote:
>
>> How about a new python 3 release with (possibly partial) backwards
>> compatibility with 2.6? I'm a big 3 fan, but I'm dismayed at the way major
>> software hasn't been ported to it. I'm eager to use 3, but paradoxically,
>> the 3 release makes me rather stuck with 2.6. Excuse me if this has been
>> suggested in the past.
>>
>>
> The proper route to take, in my opinion, is to see what 2.x libraries you
> are using that are not 3.x compatible, run 2to3 on them, then run their test
> suite, and see where you get. Submit a patch or two to the library and see
> what happens -- it at least gets the wheels in motion.
>

It's not even that easy -- libraries can't apply patches for Python 3
compatibility as they usually break Python 2 compatibility.  Potentially
libraries could apply patches that make a codebase 2to3 ready, but from what
I've seen that's more black magic than straight forward updating, as such
patches have to trick 2to3 producing the output that is desired.

The only workable workflow I've seen people propose for maintaining a single
codebase with compatibility across both 2 and 3 is to use such tricks, with
aliases to suppress some 2to3 updates when they are inappropriate, so that
you can run 2to3 on install and have a single canonical Python 2 source.
 Python 2.7 won't help much (even though it is trying) as the introduction
of non-ambiguous constructions like b"" aren't compatible with previous
versions of Python and so can't be used in many libraries (support at least
back to Python 2.5 is the norm for most libraries, I think).

Also, running 2to3 on installation is kind of annoying, as you get source
that isn't itself the canonical source, so to fix bugs you have to look at
the installed source and trace it back to the bug in the original source.

I suspect a reasonable workflow might be possible with hg and maybe patch
queues, but I don't feel familiar enough with those tools to map that out.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/e43ccd78/attachment-0007.htm>

From glyph at twistedmatrix.com  Tue Jan  5 21:24:45 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 5 Jan 2010 15:24:45 -0500
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.

From regebro at gmail.com  Tue Jan  5 21:57:35 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Jan 2010 21:57:35 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <319e029f1001051257k3827c52ahcd115b15d9a8ece7@mail.gmail.com>

On Tue, Jan 5, 2010 at 21:24, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It seems like this is a problem to be addressed, then. ?Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

Some of it is about changing the code so 2to3 doesn't have to do ugly
things. For example, always use iteritems(), so that 2to3 knows that
items() can be used without converting it to a list, etc. Then we do
have the problems with unicode vs string vs bytes, where I can't think
of a clever solution except your no-op shims, which admittedly is
fugly . For me that tends to turn into testing everything until the
tests run on all platforms, which I guess is close to "black magic".
Don't know what to do about that.

python-incompatibility is about documenting all differences, and also
how to make code that runs under both 2.6 and 3.0 without 2to3. But I
guess it should be extended into how to spell something that is clean
in both 2.6 and 3.x.

>> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.

Ian: If you have examples of 2to3 updated that are not appropriate
(except the x.items() -> list(x.items()) they would be appreciated.

> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

In my experience it turned out to be less of a problem than I thought.
That is to some extent because the modules I've ported has had good
test coverage, and I have fixed 99.9% of the bugs by making the tests
pass. Developing with Distributes 2to3 support has then been quite
smooth and in most cases the separation between the 2.x source and the
3.x source hasn't been a problem.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan  5 22:07:23 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 22:07:23 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <4B43AA0B.5030308@v.loewis.de>

> It's not even that easy -- libraries can't apply patches for Python 3
> compatibility as they usually break Python 2 compatibility.  Potentially
> libraries could apply patches that make a codebase 2to3 ready, but from
> what I've seen that's more black magic than straight forward updating,
> as such patches have to trick 2to3 producing the output that is desired.

I wouldn't qualify it in that way. It may be necessary occasionally to
trick 2to3, but that's really a bug in 2to3 which you should report, so
that trickery is then a work-around for a bug - something that you may
have to do with other API, as well.

The "black magic" is really more in the parts that 2to3 doesn't touch
at all (because they are inherently not syntactic); these are the
problem areas Guido refers to. The "black magic" then is to make the
same code work unmodified for both 2.x and 3.x.

> The only workable workflow I've seen people propose for maintaining a
> single codebase with compatibility across both 2 and 3 is to use such
> tricks, with aliases to suppress some 2to3 updates when they are
> inappropriate

I think you misunderstand. All this is necessary, but *not* to suppress
2to3 updates. More typically, it is necessary because 2to3 leaves the
code unmodified either way.

> Also, running 2to3 on installation is kind of annoying, as you get
> source that isn't itself the canonical source, so to fix bugs you have
> to look at the installed source and trace it back to the bug in the
> original source.

That's an issue indeed, but one that I would hope that can be fixed by
improved traceback printing. It would be good if such traceback printing
could make it into 2.7.

Regards,
Martin


From martin at v.loewis.de  Tue Jan  5 22:13:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 05 Jan 2010 22:13:07 +0100
Subject: [Python-Dev] Suggestion: new 3 release with
	backwards	compatibility
In-Reply-To: <34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<34F4E992-621D-4400-B7CF-865C38BB7755@twistedmatrix.com>
Message-ID: <4B43AB63.3000607@v.loewis.de>

> No-op constructions like 'bytes("")' could help for older versions of
> Python, though.  A very, very small runtime shim could provide
> support for these, if 2to3 could be told about it somehow.

You actually don't *need* 2to3 support for that - bytes("") can work
in either version:

2.x:
def bytes(s):
  return s

3.x:
def bytes(s)
  return s.encode("ascii")

On top of that, you *could* as for bytes("") to be converted to b"" in
3.x - and it is indeed possible to tell 2to3 about that, and you don't
even need to modify 2to3's source to do so: it can be part of your
package.

>> Also, running 2to3 on installation is kind of annoying, as you get
>> source that isn't itself the canonical source, so to fix bugs you
>> have to look at the installed source and trace it back to the bug
>> in the original source.
> 
> Given the way tracebacks are built, i.e. from filenames stored in
> .pycs rather than based on where the code was actually loaded in the
> filesystem, couldn't 2to3 could do .pyc rewriting to point at the
> original source?  Sort of like our own version of the #line
> directive? :)

I think it could, but it would be fairly flaky as the pycs
can get lost, and lose that information after regeneration. Also,
it may be confusing in other scenarios.

I'd rather have it generate separate mapper files, and change the
traceback printing to consider these (as an option).

> Seriously though, I find it hard to believe that this is a big
> problem.  The 3.x source looks pretty similar to the 2.x source, and
> it's good to look at both if you're dealing with a 3.x issue.

That's my experience as well - the only challenge is that you can't
cut-n-paste the source URL into an editor.

Regards,
Martin


From ianb at colorstudy.com  Tue Jan  5 22:39:21 2010
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 5 Jan 2010 15:39:21 -0600
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <4B43AA0B.5030308@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com> 
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com> 
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com> 
	<4B43AA0B.5030308@v.loewis.de>
Message-ID: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>

On Tue, Jan 5, 2010 at 3:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > It's not even that easy -- libraries can't apply patches for Python 3
> > compatibility as they usually break Python 2 compatibility. ?Potentially
> > libraries could apply patches that make a codebase 2to3 ready, but from
> > what I've seen that's more black magic than straight forward updating,
> > as such patches have to trick 2to3 producing the output that is desired.
>
> I wouldn't qualify it in that way. It may be necessary occasionally to
> trick 2to3, but that's really a bug in 2to3 which you should report, so
> that trickery is then a work-around for a bug - something that you may
> have to do with other API, as well.
>
> The "black magic" is really more in the parts that 2to3 doesn't touch
> at all (because they are inherently not syntactic); these are the
> problem areas Guido refers to. The "black magic" then is to make the
> same code work unmodified for both 2.x and 3.x.

Just to clarify, the black magic I'm referring to is things like:

try:
?? ?unicode_ = unicode
except NameError:
?? ?unicode_ = str

and some other aliases like this that are unambiguous and which 2to3
won't touch (if you write them correctly). ?If the porting guide noted
all these tricks (of which several have been developed, and I'm only
vaguely aware of a few) that would be helpful. ?It's not a lot of
tricks, but the tricks are not obvious and 2to3 gets the translation
wrong pretty often without them. ?For instance, when I say str in
Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
str and unicode to str. ?That *nothing* translates to bytes by default
(AFAIK) means that people must either be living in a bytes-free world
(which sure, lots of code does) or they are using tricks not included
in 2to3 itself.


Also replying to Glyph:
> > Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.
>
> Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in
the filesystem, couldn't 2to3 could do .pyc rewriting to point at the
original source? ?Sort of like our own version of the #line directive?
:)
>
> Seriously though, I find it hard to believe that this is a big problem. ?The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
then I don't currently develop any code that is "installed", I only
develop code that is directly from a source code checkout, and where
the checkout is put on the path.  I guess I could have something that
automatically builds the code on every edit, and that's not
infeasible.  It's just not fun.  So long as I have to support Python 2
(which is like forever) then adding Python 3 only makes development
that much more complicated and much less fun, with no concrete
benefits.  Which is a terribly crotchety of me.  Sorry.

--
Ian Bicking ?| ?http://blog.ianbicking.org ?| ?http://topplabs.org/civichacker


From martin at v.loewis.de  Tue Jan  5 23:23:53 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 05 Jan 2010 23:23:53 +0100
Subject: [Python-Dev] Suggestion: new 3 release with backwards
	compatibility
In-Reply-To: <b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
Message-ID: <4B43BBF9.2000302@v.loewis.de>

> Just to clarify, the black magic I'm referring to is things like:
> 
> try:
>     unicode_ = unicode
> except NameError:
>     unicode_ = str
> 
> and some other aliases like this that are unambiguous and which 2to3
> won't touch (if you write them correctly).  If the porting guide noted
> all these tricks (of which several have been developed, and I'm only
> vaguely aware of a few) that would be helpful.  It's not a lot of
> tricks, but the tricks are not obvious and 2to3 gets the translation
> wrong pretty often without them.  For instance, when I say str in
> Python 2 I often means bytes, unsurprisingly, but 2to3 translates both
> str and unicode to str.

No, that's not what's happening. Instead, str is not translated at all
(i.e. it's *not* translated to str - it just isn't touched).

Translating unicode to str is always correct, AFAICT.

I'm not quite sure what the magic above is supposed to achieve: in 2.x,
unicode_ becomes an alias to unicode, in 3.x, 2to3 translates unicode
to str, and then the block becomes

try:
  unicode_ = str
except NameError:
  unicode_ = str

so the NameError is actually never triggered.

Could it be that the magic is proposed for a mode where you *don't*
use 2to3?

> That *nothing* translates to bytes by default
> (AFAIK) means that people must either be living in a bytes-free world
> (which sure, lots of code does) or they are using tricks not included
> in 2to3 itself.

By your above definition of "translates", the function "bytes" is
translated to bytes (i.e. it isn't touched by 2to3).

> Since 2to3 maintains line numbers yes, it wouldn't be that bad.  But
> then I don't currently develop any code that is "installed", I only
> develop code that is directly from a source code checkout, and where
> the checkout is put on the path.  I guess I could have something that
> automatically builds the code on every edit, and that's not
> infeasible.  It's just not fun.

Depends on where you draw your fun from. It is indeed fun to me, but
I can see why it may not be fun to you. So you could ask me to do it for
you - or you may try to use what I have already done for you, so you
don't have to do it.

> So long as I have to support Python 2
> (which is like forever) then adding Python 3 only makes development
> that much more complicated and much less fun, with no concrete
> benefits.

I doubt it will be *much* more complicated - only a little.
As for concrete benefits: there may be no benefits at this point,
but in the long run, starting now will have saved you a lot of
pressure in the long run, and stop users from switching away from
your packages because of lack of Python 3 support.

Regards,
Martin


From ziade.tarek at gmail.com  Wed Jan  6 01:08:34 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 01:08:34 +0100
Subject: [Python-Dev] PEP 386 and PEP 345
Message-ID: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>

Hi,

I think we've reached a consensus on those two PEPs.

Although, there's one last point that was forgotten in the discussions
: I've introduced "rc" in the pre-releases markers, so PEP 386 is
compatible with Python's own version scheme.  "rc" comes right after
"c" in the sorting. It's slightly redundant with the "c" marker but I
don't think this really matters as long as consumers know how to order
them (a < b < c < rc). I have also stated that "c" is the preferred
marker for third party projects, from PEP 386 point of view.

Is there anything else I can do to make those two PEPs accepted ?

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From david.lyon at preisshare.net  Wed Jan  6 01:26:34 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:26:34 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <4B43BBF9.2000302@v.loewis.de>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
	<4B43AA0B.5030308@v.loewis.de>
	<b654cd2e1001051339h46a7a7c7q7368f872ac45890e@mail.gmail.com>
	<4B43BBF9.2000302@v.loewis.de>
Message-ID: <1322.218.214.45.58.1262737594.squirrel@syd-srv02.ezyreg.com>

Hi Martin,

> ... but in the long run, starting now will have saved you a lot of
> pressure in the long run, and stop users from switching away from
> your packages because of lack of Python 3 support.

In a production situation it works the other way around. If there's
an application that requires twisted (or whatever package) then most
people would use the appropriate interpreter to match the library.

Since you guys all did your jobs so well :-) doing so is painless.

Because there is so much "comfort" with the existing situation it
makes it an effort for people to move to a different lounge chair.
Namely python 3.

I'd suggest that moving the package set (pypi) to python 3 could
be kicked along with the help of some automated tools. I don't
know what tools you guys have got.

But I am very sure that if code analysis was provided to package
developers on python 3 (so they don't have to run it themselves),
then it would be like an even bigger tv screen in a bigger lounge
room and it would assist in drawing them over.

David







From david.lyon at preisshare.net  Wed Jan  6 01:36:24 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 11:36:24 +1100 (EST)
Subject: [Python-Dev] Suggestion: new 3 release with backwards
 compatibility
In-Reply-To: <b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
References: <1fb8b0211001050810q4d054a10v23593ab83368c3b4@mail.gmail.com>
	<cf9f31f21001050921w82ffe7eme58f72bd187c087d@mail.gmail.com>
	<b654cd2e1001051100h14c2d6fai431bdb8d6d40b411@mail.gmail.com>
Message-ID: <1337.218.214.45.58.1262738184.squirrel@syd-srv02.ezyreg.com>


Hi Ian,

> The only workable workflow I've seen people propose for maintaining a
> single
> codebase with compatibility across both 2 and 3 is to use such tricks,
> with
> aliases to suppress some 2to3 updates when they are inappropriate, so that
> you can run 2to3 on install and have a single canonical Python 2 source.
>  Python 2.7 won't help much (even though it is trying) as the introduction
> of non-ambiguous constructions like b"" aren't compatible with previous
> versions of Python and so can't be used in many libraries (support at
> least
> back to Python 2.5 is the norm for most libraries, I think).
>
> Also, running 2to3 on installation is kind of annoying, as you get source
> that isn't itself the canonical source, so to fix bugs you have to look at
> the installed source and trace it back to the bug in the original source.
>
> I suspect a reasonable workflow might be possible with hg and maybe patch
> queues, but I don't feel familiar enough with those tools to map that out.

That's why I am saying that we need a Code-Repository: section in PEP-345
Metadata.

Let package developers keep two branches in SCM. One for 2.x and one for 3.x

Let them backport features from their 3.x development series to their 2.x
code base. In exactly the same way that it is done in so many other
developments today.

Keeping multiple branches of code is a very common thing these days. And
there are tools in the outside world (bzr,svn,mercurial) that allow us to
do it very easily.

Let's have that support in python; it will make life easier.

David










From david.lyon at preisshare.net  Wed Jan  6 03:01:22 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 6 Jan 2010 13:01:22 +1100 (EST)
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <1617.218.214.45.58.1262743282.squirrel@syd-srv02.ezyreg.com>

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?

Tarek,

Given that I helped out so much last year with the PEP in discussing
different options, even if they weren't accepted, I really feel that it is
unfair if my name isn't mentioned. It was a huge time sacrifice on my
part.

For example, even if I only managed to explain the version numbering and
clarify how that worked. It did take me some time to do that.

What I did do however, was spend a lot of time with the multiplatform
"Markers". I still think that two short weeks more of "discussion" could
resolve some issues. That discussion went for 4 months on distutils-ml.

Look, major issues aside, can you make the following concessions on
PEP-345 which I only feel will help it, namely:

 1) Source-Repository: specify a code repository to install from

 2) Streamline Requires-Python: by implementing "Markers" as
    noted by the PEP. A marker being something like
    "Requires-Python(windows): lxml". Otherwise remove the
    word marker from the PEP and just replace with "metacode".
    Markers are what were discussed on distutils-ml. Metacode
    is what is described in the PEP.

 3) Remove the inconsistency and platform ambiguities. Especially
    for windows users. For example, "win32" is extremely confusing
    for windows users right now. As more and more systems now are
    64 bit. Use the platform module, instead of the sys module
    for constants. I'll post to distutils-ml on this.

I am certainly not trying to hold this PEP up, and I apologise on
my part for my late attention. I will post to distutils-ml on these
and i promise to keep my comments unheated and unwitty.

Having said that, PEP-345 is *super-important*. A week or two or three
more discussion and the issues can be resolved.

We all just want to focus on being productive. It would be a great
accomplishment for you to get PEP-345 approved and likewise for
me getting mentioned even in a minor role as helping out on a PEP.

So I'm hoping that you can make a few last minute concessions
meaning that we can all happily go on our way in 2010.

Regards

David













From brett at python.org  Wed Jan  6 06:20:30 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Jan 2010 21:20:30 -0800
Subject: [Python-Dev] PEP 386 and PEP 345
In-Reply-To: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
References: <94bdd2611001051608p421d73bjbaa5c4f831efac5c@mail.gmail.com>
Message-ID: <bbaeab101001052120qe888df8o7eb03a61e6c030c7@mail.gmail.com>

On Tue, Jan 5, 2010 at 16:08, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hi,
>
> I think we've reached a consensus on those two PEPs.
>
> Although, there's one last point that was forgotten in the discussions
> : I've introduced "rc" in the pre-releases markers, so PEP 386 is
> compatible with Python's own version scheme.  "rc" comes right after
> "c" in the sorting. It's slightly redundant with the "c" marker but I
> don't think this really matters as long as consumers know how to order
> them (a < b < c < rc). I have also stated that "c" is the preferred
> marker for third party projects, from PEP 386 point of view.
>
> Is there anything else I can do to make those two PEPs accepted ?
>

As you said, consensus has been reached, so just Guido's BDFL stamp of
approval is all I can think of.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100105/a76cb75b/attachment-0007.htm>

From chris at simplistix.co.uk  Wed Jan  6 12:19:45 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:19:45 +0000
Subject: [Python-Dev] bug triage
Message-ID: <4B4471D1.9020707@simplistix.co.uk>

Hi All,

Is there a high volume of incoming bugs to the Python tracker?
If so, I'd like to help with triaging. I think I have all the necessary 
access, what I'm missing is the knowledge of how to set myself up to get 
notifications of new bugs...

How do I do that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From fuzzyman at voidspace.org.uk  Wed Jan  6 12:24:37 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 06 Jan 2010 11:24:37 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <4B4472F5.8000709@voidspace.org.uk>

On 06/01/2010 11:19, Chris Withers wrote:
> Hi All,
>
> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the 
> necessary access, what I'm missing is the knowledge of how to set 
> myself up to get notifications of new bugs...
>
> How do I do that?
>

Bug triaging is one of Python's "big needs" and anything you do to help 
on this score would be much appreciated. Particularly reviewing new and 
outstanding issues.

I assumed there would be RSS feeds for bug tracker activity but can't 
easily find these on the tracker. There is a bot that posts activity to 
#python-dev, so there must be some way of getting this information.

All the best,

Michael



> cheers,
>
> Chris
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From solipsis at pitrou.net  Wed Jan  6 12:25:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 11:25:16 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <loom.20100106T122258-896@post.gmane.org>


Hi Chris,

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary 
> access, what I'm missing is the knowledge of how to set myself up to get 
> notifications of new bugs...

Do you really want to get such notifications? There may be a lot of them.
If you want however, you can join #python-dev on IRC (irc.freenode.net) where
there's a bot which posts updates of all bugs on the tracker. There's usually
not a lot of discussion going on so you probably won't feel flooded.

In addition to bug triage, what is needed is reviewing of existing patches, as
well as writing patches for issues which haven't been addressed yet :-)

Regards

Antoine.




From chris at simplistix.co.uk  Wed Jan  6 12:30:40 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:30:40 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B447460.7080100@simplistix.co.uk>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't 
> easily find these on the tracker. There is a bot that posts activity to 
> #python-dev, so there must be some way of getting this information.

Yeah, email-out is what I'm really after... I have it for my own Roundup 
instance so it can't be that hard to do ;-)

Roch?, you guys host the bug tracker, right? Is there email-out set up 
for it?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ncoghlan at gmail.com  Wed Jan  6 12:37:23 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 21:37:23 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <4B4475F3.5040406@gmail.com>

Michael Foord wrote:
> I assumed there would be RSS feeds for bug tracker activity but can't
> easily find these on the tracker. There is a bot that posts activity to
> #python-dev, so there must be some way of getting this information.

I'm pretty sure the bugs list is still the primary spooled notification
mechanism:
http://mail.python.org/mailman/listinfo/python-bugs-list

There are also the weekly tracker activity summaries that are posted
here to python-dev.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From chris at simplistix.co.uk  Wed Jan  6 12:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 11:41:28 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <4B4476E8.8050709@simplistix.co.uk>

Nick Coghlan wrote:
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

That's what I was after, thanks!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From facundobatista at gmail.com  Wed Jan  6 13:03:08 2010
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 6 Jan 2010 09:03:08 -0300
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4471D1.9020707@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
Message-ID: <e04bdf311001060403y3b65c647jf82fdc26266a0efe@mail.gmail.com>

On Wed, Jan 6, 2010 at 8:19 AM, Chris Withers <chris at simplistix.co.uk> wrote:

> Is there a high volume of incoming bugs to the Python tracker?
> If so, I'd like to help with triaging. I think I have all the necessary
> access, what I'm missing is the knowledge of how to set myself up to get
> notifications of new bugs...

Not notifications, but maybe a way to have a higher look of them for
easy selection:

  http://www.taniquetil.com.ar/cgi-bin/pytickets.py

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ziade.tarek at gmail.com  Wed Jan  6 13:29:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:29:58 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4472F5.8000709@voidspace.org.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
Message-ID: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>

On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 06/01/2010 11:19, Chris Withers wrote:
>>
>> Hi All,
>>
>> Is there a high volume of incoming bugs to the Python tracker?
>> If so, I'd like to help with triaging. I think I have all the necessary
>> access, what I'm missing is the knowledge of how to set myself up to get
>> notifications of new bugs...
>>
>> How do I do that?
>>
>
> Bug triaging is one of Python's "big needs" and anything you do to help on
> this score would be much appreciated. Particularly reviewing new and
> outstanding issues.

Another useful triage I think, is to review the oldest bugs (some of
them are > 5 years)
and remove the ones that are not relevant anymore, or duplicate with
newer entries.

Tarek

-- 
Tarek Ziad? | http://ziade.org


From chris at simplistix.co.uk  Wed Jan  6 13:31:08 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 06 Jan 2010 12:31:08 +0000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B44828C.4070201@simplistix.co.uk>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this 
with someone as a paired task for those 2 days...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Wed Jan  6 13:37:55 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 6 Jan 2010 13:37:55 +0100
Subject: [Python-Dev] bug triage
In-Reply-To: <4B44828C.4070201@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B44828C.4070201@simplistix.co.uk>
Message-ID: <94bdd2611001060437s2d0242aembc669c3263773fea@mail.gmail.com>

On Wed, Jan 6, 2010 at 1:31 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Another useful triage I think, is to review the oldest bugs (some of
>> them are > 5 years)
>> and remove the ones that are not relevant anymore, or duplicate with
>> newer entries.
>
> I'm sprinting for 2 days at PyCon, I'd verymuch be up for doing this with
> someone as a paired task for those 2 days...

I'll be doing Distutils stuff but I can probably help around a bit in that task:
Distutils have quite a few old issues so I can tackle those


From roche at upfrontsystems.co.za  Wed Jan  6 13:32:39 2010
From: roche at upfrontsystems.co.za (=?ISO-8859-1?Q?Roch=E9?= Compaan)
Date: Wed, 06 Jan 2010 14:32:39 +0200
Subject: [Python-Dev] bug triage
In-Reply-To: <4B447460.7080100@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B447460.7080100@simplistix.co.uk>
Message-ID: <1262781159.2836.218.camel@didi>

On Wed, 2010-01-06 at 11:30 +0000, Chris Withers wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't 
> > easily find these on the tracker. There is a bot that posts activity to 
> > #python-dev, so there must be some way of getting this information.
> 
> Yeah, email-out is what I'm really after... I have it for my own Roundup 
> instance so it can't be that hard to do ;-)
> 
> Roch?, you guys host the bug tracker, right? Is there email-out set up 
> for it?

We do, but we don't administer it. There are a few administrators taking
care of it and you should be able to reach them by logging your request
here: http://psf.upfronthosting.co.za/roundup/meta/ or post it to the
Infrastructure mailing list: infrastructure at python.org


-- 
Roch? Compaan
Upfront Systems                   http://www.upfrontsystems.co.za



From ncoghlan at gmail.com  Wed Jan  6 13:57:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 06 Jan 2010 22:57:24 +1000
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <4B4488B4.2070208@gmail.com>

Tarek Ziad? wrote:
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I believe someone (Daniel Diniz, maybe?) did do a pass over those some
time in the  last 12 months, so most of the obviously irrelevant ones
that are that old should already be gone. Not to say it isn't worth
doing another pass, just saying not to get disheartened if there aren't
many that can be readily closed.

There are at least a few still kicking around just because they're
difficult to deal with (there's an ancient one to do with one of the
ways circular imports can fail that I occasionally go back and reread
before moving on to something more tractable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rdmurray at bitdance.com  Wed Jan  6 14:43:17 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 08:43:17 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4476E8.8050709@simplistix.co.uk>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
	<4B4476E8.8050709@simplistix.co.uk>
Message-ID: <20100106134317.3EF141FB5A6@kimball.webabinitio.net>

On Wed, 06 Jan 2010 11:41:28 +0000, Chris Withers <chris at simplistix.co.uk> wrote:
> Nick Coghlan wrote:
> > I'm pretty sure the bugs list is still the primary spooled notification
> > mechanism:
> > http://mail.python.org/mailman/listinfo/python-bugs-list
> 
> That's what I was after, thanks!

Just for completeness, there's also new-bugs-announce if you want
just *new* bug notification.  That's more for people who want to
watch for bugs they want to become nosy on, though; if you are
doing triage python-bugs-list is what you want.

Please also read http://www.python.org/dev/workflow/ if you haven't
already.  Thanks for being willing to chip in!

--David


From brian.curtin at gmail.com  Wed Jan  6 15:57:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 08:57:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4488B4.2070208@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
Message-ID: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I believe someone (Daniel Diniz, maybe?) did do a pass over those some
> time in the  last 12 months, so most of the obviously irrelevant ones
> that are that old should already be gone. Not to say it isn't worth
> doing another pass, just saying not to get disheartened if there aren't
> many that can be readily closed.
>
> There are at least a few still kicking around just because they're
> difficult to deal with (there's an ancient one to do with one of the
> ways circular imports can fail that I occasionally go back and reread
> before moving on to something more tractable).
>
> Cheers,
> Nick.
>

On the topic of bugs that can be readily closed (literally), I've recently
come across a number of issues which appear to be sitting in a patch or
review stage, but their patches have been committed and the issue remains
open. What is the best course of action there? I'd just go ahead and close
the issue myself but I don't have tracker privileges.

I'm willing to help out with another Daniel Diniz-esque triage sweep if that
would help.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/05d9ebd7/attachment-0007.htm>

From ssteinerx at gmail.com  Wed Jan  6 16:14:20 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 6 Jan 2010 10:14:20 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
Message-ID: <7FCF8B19-3465-4392-80CC-BEAEBD926ECA@gmail.com>


On Jan 6, 2010, at 7:29 AM, Tarek Ziad? wrote:

> On Wed, Jan 6, 2010 at 12:24 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> On 06/01/2010 11:19, Chris Withers wrote:
>>> 
>>> Hi All,
>>> 
>>> Is there a high volume of incoming bugs to the Python tracker?
>>> If so, I'd like to help with triaging. I think I have all the necessary
>>> access, what I'm missing is the knowledge of how to set myself up to get
>>> notifications of new bugs...
>>> 
>>> How do I do that?
>>> 
>> 
>> Bug triaging is one of Python's "big needs" and anything you do to help on
>> this score would be much appreciated. Particularly reviewing new and
>> outstanding issues.
> 
> Another useful triage I think, is to review the oldest bugs (some of
> them are > 5 years)
> and remove the ones that are not relevant anymore, or duplicate with
> newer entries.

I was actually thinking about that the other day when I saw that the average age of bugs on the Python tracker was at some hideously large 3 digit number.  

The 'success' statistic would be to bring that down below, say, 100.

S



From skip at pobox.com  Wed Jan  6 17:38:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 6 Jan 2010 10:38:13 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <4B4475F3.5040406@gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <19268.48245.616046.874126@montanaro.dyndns.org>

>>>>> "Nick" == Nick Coghlan <ncoghlan at gmail.com> writes:

    Nick> I'm pretty sure the bugs list is still the primary spooled
    Nick> notification mechanism:
    Nick> http://mail.python.org/mailman/listinfo/python-bugs-list

Actually, there is a new-bugs-announce list:

    http://mail.python.org/mailman/listinfo/new-bugs-announce

Skip


From solipsis at pitrou.net  Wed Jan  6 18:56:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 17:56:49 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <hi2it0$q48$1@ger.gmane.org>

Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> On the topic of bugs that can be readily closed (literally), I've
> recently come across a number of issues which appear to be sitting in a
> patch or review stage, but their patches have been committed and the
> issue remains open. What is the best course of action there?

Post a message on the issue asking for info.




From solipsis at pitrou.net  Wed Jan  6 19:09:39 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 6 Jan 2010 18:09:39 +0000 (UTC)
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<hi2it0$q48$1@ger.gmane.org>
Message-ID: <loom.20100106T190650-337@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> Le Wed, 06 Jan 2010 08:57:42 -0600, Brian Curtin a ?crit?:
> > On the topic of bugs that can be readily closed (literally), I've
> > recently come across a number of issues which appear to be sitting in a
> > patch or review stage, but their patches have been committed and the
> > issue remains open. What is the best course of action there?
> 
> Post a message on the issue asking for info.

Ok, I realize my answer might have been a bit terse :-)
The patch might be waiting to be merged in all development branches, or it may
not totally resolve the issue, or perhaps documentation needs to be updated, or
perhaps it is pending a verdict from the buildbots, etc. You can't deduce that
the issue is completely fixed from the simple fact that something has been
committed.

Regards

Antoine Pitrou.






From brett at python.org  Wed Jan  6 20:03:32 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 11:03:32 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
Message-ID: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>

On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Wed, Jan 6, 2010 at 06:57, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>  I believe someone (Daniel Diniz, maybe?) did do a pass over those some
>> time in the  last 12 months, so most of the obviously irrelevant ones
>> that are that old should already be gone. Not to say it isn't worth
>> doing another pass, just saying not to get disheartened if there aren't
>> many that can be readily closed.
>>
>> There are at least a few still kicking around just because they're
>> difficult to deal with (there's an ancient one to do with one of the
>> ways circular imports can fail that I occasionally go back and reread
>> before moving on to something more tractable).
>>
>> Cheers,
>> Nick.
>>
>
> On the topic of bugs that can be readily closed (literally), I've recently
> come across a number of issues which appear to be sitting in a patch or
> review stage, but their patches have been committed and the issue remains
> open. What is the best course of action there? I'd just go ahead and close
> the issue myself but I don't have tracker privileges.
>
>
If a core developer is willing to step forward and vouch for you to get
tracker privileges then I will give them to you. We are trying to give out
tracker privs w/ less time than required to get commit privileges. So as
long as you have helped out on a few issues in a positive and correct way
that should be enough to get one of the regulars who perform triage to
notice.

-Brett



I'm willing to help out with another Daniel Diniz-esque triage sweep if that
> would help.
>
> Brian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/f24c9595/attachment-0007.htm>

From nad at acm.org  Wed Jan  6 21:41:05 2010
From: nad at acm.org (Ned Deily)
Date: Wed, 06 Jan 2010 12:41:05 -0800
Subject: [Python-Dev] bug triage
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> <4B4475F3.5040406@gmail.com>
Message-ID: <nad-19B167.12410506012010@news.gmane.org>

In article <4B4475F3.5040406 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
> > I assumed there would be RSS feeds for bug tracker activity but can't
> > easily find these on the tracker. There is a bot that posts activity to
> > #python-dev, so there must be some way of getting this information.
> 
> I'm pretty sure the bugs list is still the primary spooled notification
> mechanism:
> http://mail.python.org/mailman/listinfo/python-bugs-list

Also, that mailing list (along with most python development related 
mailing lists) is mirrored at gmane.org which means it can also be 
obtained via a newsreader (NNTP) or various RSS feeds.

http://dir.gmane.org/gmane.comp.python.bugs

-- 
 Ned Deily,
 nad at acm.org



From nick.bastin at gmail.com  Wed Jan  6 22:14:54 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:14:54 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
Message-ID: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>

(This may occur on more platforms - I can test on more unix platforms
if the consensus is this is an actual problem and I'm not just a nut)

On freebsd5, if you do a simple ./configure --enable-shared in current
(2.7) trunk, your python shared library will build properly, but all
modules will fail to find the shared library and thus fail to build:

gcc -shared build/temp.freebsd-5.3-RELEASE-i386-2.7/u1/Python/Python-2.7a1/Modules/_struct.o
   -L/u1/tmp/python2.7a1/lib -L/usr/local/lib -lpython2.7 -o
build/lib.freebsd-5.3-RELEASE-i386-2.7/_struct.so
/usr/bin/ld: cannot find -lpython2.7
building '_ctypes_test' extension
...

This of course is because libpython2.7.so is in the current directory
and not (yet) installed in /usr/local/lib.  I've made a very simple
fix for this problem that works, but at least to me smells a bit
funny, which is to modify setup.py to add the following to
detect_modules():

        # If we did --enable-shared, we need to be able to find the library
        # we just built in order to build the modules.
        if platform == 'freebsd5':
            add_dir_to_list(self.compiler_obj.library_dirs, '.')


Which brings me to a few questions:

a) Does this seem like a real problem, or am I missing something obvious?

b) Does this fix seem like the sensible thing to do?  (it seems at
least that we ought to check that the user configured --enable-shared
and only set -L. in that case, if that's possible)

Setting --enable-shared when you actually have a libpython2.7.so in
/usr/local/lib (or whatever --prefix you've selected) is possibly even
more dangerous, because it may succeed in linking against a
differently-built library than what you intended.

--
Nick


From nick.bastin at gmail.com  Wed Jan  6 22:39:17 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 16:39:17 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <66d0a6e11001061339t38202b70mca1091e1926ecf4b@mail.gmail.com>

On Wed, Jan 6, 2010 at 16:14, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> This of course is because libpython2.7.so is in the current directory
> and not (yet) installed in /usr/local/lib.

One minor correction - as you could see from the compile line, the
actual --prefix in this case is /u1/tmp/python2.7a1, but the libraries
obviously aren't installed there yet either.  Perhaps a better fix
than setting -L. would be to put the shared library in
build/lib.freebsd-5.3-RELEASE-i386-2.7 and add that to the library
path for the linker (the build creates this directory, but installs
nothing in it).

--
Nick


From martin at v.loewis.de  Wed Jan  6 23:21:44 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 06 Jan 2010 23:21:44 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
Message-ID: <4B450CF8.8090009@v.loewis.de>

> b) Does this fix seem like the sensible thing to do?

No. Linking in setup.py should use the same options as if the module
was built as *shared* through Modules/Setup, which, IIUC, should use
BLDLIBRARY.

Regards,
Martin


From nick.bastin at gmail.com  Thu Jan  7 01:08:13 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 6 Jan 2010 19:08:13 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B450CF8.8090009@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
Message-ID: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:21, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> b) Does this fix seem like the sensible thing to do?
>
> No. Linking in setup.py should use the same options as if the module
> was built as *shared* through Modules/Setup, which, IIUC, should use
> BLDLIBRARY.

Thanks for that pointer, that makes much more sense.  Indeed,
BLDLIBRARY on FreeBSD* is set to '-L. -lpython$(VERSION)' if you set
--enable-shared, but somehow that piece of information doesn't
propagate into the module build.  More investigation to be done...

--
Nick


From rdmurray at bitdance.com  Thu Jan  7 02:22:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Jan 2010 20:22:42 -0500
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
Message-ID: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>


On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On the topic of bugs that can be readily closed (literally), I've recently
> > come across a number of issues which appear to be sitting in a patch or
> > review stage, but their patches have been committed and the issue remains
> > open. What is the best course of action there? I'd just go ahead and close
> > the issue myself but I don't have tracker privileges.
> >
> >
> If a core developer is willing to step forward and vouch for you to get
> tracker privileges then I will give them to you. We are trying to give out
> tracker privs w/ less time than required to get commit privileges. So as
> long as you have helped out on a few issues in a positive and correct way
> that should be enough to get one of the regulars who perform triage to
> notice.
> 
> -Brett

I've done a quick scan of issues Brian is nosy on to refresh my
memory, and I'd say he's definitely been making positive contributions.
I'm willing to volunteer to keep an eye on his triage work for a while
if you grant him tracker privs.

Brian, I assume you'll be cognizant of Antoine's advice about making
sure a bug really should be closed before closing it :)  Hanging out in
#python-dev on freenode while working on issues can be helpful, as well,
since you can quickly ask whoever is there for second opinions on
particular bugs.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Thu Jan  7 02:28:26 2010
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Jan 2010 17:28:26 -0800
Subject: [Python-Dev] bug triage
In-Reply-To: <20100107012242.2C7D11FBB50@kimball.webabinitio.net>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk> 
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com> 
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com> 
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com> 
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
Message-ID: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>

On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com> wrote:

>
> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On the topic of bugs that can be readily closed (literally), I've
> recently
> > > come across a number of issues which appear to be sitting in a patch or
> > > review stage, but their patches have been committed and the issue
> remains
> > > open. What is the best course of action there? I'd just go ahead and
> close
> > > the issue myself but I don't have tracker privileges.
> > >
> > >
> > If a core developer is willing to step forward and vouch for you to get
> > tracker privileges then I will give them to you. We are trying to give
> out
> > tracker privs w/ less time than required to get commit privileges. So as
> > long as you have helped out on a few issues in a positive and correct way
> > that should be enough to get one of the regulars who perform triage to
> > notice.
> >
> > -Brett
>
> I've done a quick scan of issues Brian is nosy on to refresh my
> memory, and I'd say he's definitely been making positive contributions.
> I'm willing to volunteer to keep an eye on his triage work for a while
> if you grant him tracker privs.
>
>
Done for the username brian.curtin (email doesn't match the one Brian
emailed from so do let me know, Brian if this is the right username).
Welcome aboard!

-Brett


> Brian, I assume you'll be cognizant of Antoine's advice about making
> sure a bug really should be closed before closing it :)  Hanging out in
> #python-dev on freenode while working on issues can be helpful, as well,
> since you can quickly ask whoever is there for second opinions on
> particular bugs.
>
> --
> R. David Murray                                      www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/726f5ca0/attachment-0007.htm>

From brian.curtin at gmail.com  Thu Jan  7 02:59:42 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 6 Jan 2010 19:59:42 -0600
Subject: [Python-Dev] bug triage
In-Reply-To: <bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
References: <4B4471D1.9020707@simplistix.co.uk>
	<4B4472F5.8000709@voidspace.org.uk>
	<94bdd2611001060429q19287b06i51cbf51cfa43f582@mail.gmail.com>
	<4B4488B4.2070208@gmail.com>
	<cf9f31f21001060657p13331b4btac2ffbf620da4cf7@mail.gmail.com>
	<bbaeab101001061103k5624996cq43fe8e181dcc4be3@mail.gmail.com>
	<20100107012242.2C7D11FBB50@kimball.webabinitio.net>
	<bbaeab101001061728o58027773h57250943f742e46a@mail.gmail.com>
Message-ID: <cf9f31f21001061759m4fee1cc7g2d9fe8bbfd3cb38e@mail.gmail.com>

On Wed, Jan 6, 2010 at 19:28, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, Jan 6, 2010 at 17:22, R. David Murray <rdmurray at bitdance.com>wrote:
>
>>
>> On Wed, 06 Jan 2010 11:03:32 -0800, Brett Cannon wrote:
>> > On Wed, Jan 6, 2010 at 06:57, Brian Curtin <brian.curtin at gmail.com>
>> wrote:
>> > > On the topic of bugs that can be readily closed (literally), I've
>> recently
>> > > come across a number of issues which appear to be sitting in a patch
>> or
>> > > review stage, but their patches have been committed and the issue
>> remains
>> > > open. What is the best course of action there? I'd just go ahead and
>> close
>> > > the issue myself but I don't have tracker privileges.
>> > >
>> > >
>> > If a core developer is willing to step forward and vouch for you to get
>> > tracker privileges then I will give them to you. We are trying to give
>> out
>> > tracker privs w/ less time than required to get commit privileges. So as
>> > long as you have helped out on a few issues in a positive and correct
>> way
>> > that should be enough to get one of the regulars who perform triage to
>> > notice.
>> >
>> > -Brett
>>
>> I've done a quick scan of issues Brian is nosy on to refresh my
>> memory, and I'd say he's definitely been making positive contributions.
>> I'm willing to volunteer to keep an eye on his triage work for a while
>> if you grant him tracker privs.
>>
>>
> Done for the username brian.curtin (email doesn't match the one Brian
> emailed from so do let me know, Brian if this is the right username).
> Welcome aboard!
>
>
Yep, that's the one. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100106/62a95fa0/attachment-0007.htm>

From python at mrabarnett.plus.com  Thu Jan  7 04:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 07 Jan 2010 03:07:56 +0000
Subject: [Python-Dev] GIL required for _all_ Python calls?
Message-ID: <4B45500C.8090905@mrabarnett.plus.com>

Hi,

I've been wondering whether it's possible to release the GIL in the
regex engine during matching.

I know that it needs to have the GIL during memory-management calls, but
does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
an easy way to find out? Or is it just a case of checking the source
files for mentions of the GIL? The header file for PyList_New, for
example, doesn't mention it!

Thanks


From john.arbash.meinel at gmail.com  Thu Jan  7 04:25:48 2010
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Wed, 06 Jan 2010 21:25:48 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B45543C.2090200@gmail.com>

MRAB wrote:
> Hi,
> 
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out? Or is it just a case of checking the source
> files for mentions of the GIL? The header file for PyList_New, for
> example, doesn't mention it!
> 
> Thanks

Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
get concurrent updating of the value, and then the final value is wrong.
(two threads do 5+1 getting 6, rather than 7, and when the decref, you
end up at 4 rather than back at 5).

AFAIK, the only things that don't require the GIL are macro functions,
like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
example, will be increfing and setting the exception state, so certainly
needs the GIL to be held.

John
=:->



From benjamin at python.org  Thu Jan  7 04:32:17 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 6 Jan 2010 21:32:17 -0600
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45543C.2090200@gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com> <4B45543C.2090200@gmail.com>
Message-ID: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>

2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> Anything that Py_INCREF or Py_DECREF's should have the GIL, or you may
> get concurrent updating of the value, and then the final value is wrong.
> (two threads do 5+1 getting 6, rather than 7, and when the decref, you
> end up at 4 rather than back at 5).

Correct.

>
> AFAIK, the only things that don't require the GIL are macro functions,
> like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> example, will be increfing and setting the exception state, so certainly
> needs the GIL to be held.

As a general rule, I would say, no Py* macros are safe without the gil
either (the exception being Py_END_ALLOW_THREADS), since they mutate
Python objects which must be protected.



-- 
Regards,
Benjamin


From guido at python.org  Thu Jan  7 05:29:03 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 6 Jan 2010 20:29:03 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B45543C.2090200@gmail.com> 
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
Message-ID: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>

On Wed, Jan 6, 2010 at 7:32 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/1/6 John Arbash Meinel <john.arbash.meinel at gmail.com>:
> > AFAIK, the only things that don't require the GIL are macro functions,
> > like PyString_AS_STRING or PyTuple_SET_ITEM. PyErr_SetString, for
> > example, will be increfing and setting the exception state, so certainly
> > needs the GIL to be held.
>
> As a general rule, I would say, no Py* macros are safe without the gil
> either (the exception being Py_END_ALLOW_THREADS), since they mutate
> Python objects which must be protected.

That's keeping it on the safe side, since there are some macros like
PyString_AS_STRING() that are also safe, *if* you are owning at least
one reference to the string object.

At the same time, "no Py* macros" is not quite strong enough, since if
you called PyString_AS_STRING() before releasing the GIL but you don't
own a reference to the string object, the string might be deallocated
behind your back by another thread.

A better rule would be "you may access the memory buffer in a PyString
or PyUnicode object with the GIL released as long as you own a
reference to the string object." Everything else is out of bounds (or
not worth the bother).

--
--Guido van Rossum (python.org/~guido)


From yoann.padioleau at facebook.com  Thu Jan  7 08:24:42 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Wed, 6 Jan 2010 23:24:42 -0800
Subject: [Python-Dev] relation between Python.asdl and Tools/compiler/ast.txt
Message-ID: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>

Hi,

I would like to use astgen.py to generate python classes corresponding to the 
AST of something I have defined in a .asdl file, along the line of what is
apparently done for the python AST itself. I thought astgen.py would
take as an argument a .asdl file, but apparently it instead process a file
called ast.txt. Where does this file come from ? Is it generated from
Python.asdl ?



From johan.gill at agama.tv  Thu Jan  7 10:46:52 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 10:46:52 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
Message-ID: <4B45AD8C.5000405@agama.tv>

Hi devs,
the company where I work has done some work on Python, and the question 
is how this work, owned by the company, can be contributed to the 
community properly. Are there any license issues or other pitfalls we 
need to think about? I imagine that other companies have contributed 
before, so this is probably an already solved problem.

Regards
Johan Gill
Agama Technologies



From regebro at gmail.com  Thu Jan  7 12:12:17 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 12:12:17 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45AD8C.5000405@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
Message-ID: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
> Hi devs,
> the company where I work has done some work on Python, and the question is
> how this work, owned by the company, can be contributed to the community
> properly. Are there any license issues or other pitfalls we need to think
> about? I imagine that other companies have contributed before, so this is
> probably an already solved problem.

I'm not a license lawyer, but typically your company needs to give the
code to the community. Yes, it means it stops owning it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From hodgestar+pythondev at gmail.com  Thu Jan  7 12:28:01 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 7 Jan 2010 13:28:01 +0200
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <fb73205e1001070328j44ac747fu7232a89b559ad0d9@mail.gmail.com>

On Thu, Jan 7, 2010 at 1:12 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

This is incorrect.

The correct information is at http://www.python.org/psf/contrib/.

Schiavo
Simon


From swamy.sangamesh at gmail.com  Thu Jan  7 11:46:59 2010
From: swamy.sangamesh at gmail.com (swamy sangamesh)
Date: Thu, 7 Jan 2010 16:16:59 +0530
Subject: [Python-Dev] test_ctypes failure on AIX 5.3 using python-2.6.2 and
	libffi-3.0.9
Message-ID: <c3369a531001070246s441e159cg35b4cab4e3f65541@mail.gmail.com>

Hi All,

I built the python-2.6.2 with the latest libffi-3.0.9 in AIX 5.3 using xlc
compiler.
When i try to run the ctypes test cases, two failures are seen in
test_bitfields.

*test_ints (ctypes.test.test_bitfields.C_Test) ... FAIL
test_shorts (ctypes.test.test_bitfields.C_Test) ... FAIL*

I have attached the full test case result.

If i change the type c_int and c_short to c_unit and c_ushort of class
"BITS(Structure)" in file
test_bitfields.py then no failures are seen.

Has anyone faced the similar issue or any help is appreciated.


Thanks,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0007.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctype-testcases
Type: application/octet-stream
Size: 22996 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/14588c00/attachment-0007.obj>

From ncoghlan at gmail.com  Thu Jan  7 13:23:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 07 Jan 2010 22:23:11 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
Message-ID: <4B45D22F.40404@gmail.com>

Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 10:46, Johan Gill <johan.gill at agama.tv> wrote:
>> Hi devs,
>> the company where I work has done some work on Python, and the question is
>> how this work, owned by the company, can be contributed to the community
>> properly. Are there any license issues or other pitfalls we need to think
>> about? I imagine that other companies have contributed before, so this is
>> probably an already solved problem.
> 
> I'm not a license lawyer, but typically your company needs to give the
> code to the community. Yes, it means it stops owning it.

As Simon pointed out, while some organisations do work that way, the PSF
isn't one of them.

The PSF only requires that the code be contributed under a license that
then allows us to turn around and redistribute it under a different open
source license without requesting additional permission from the
copyright holder. For corporate contributions, I believe the contributor
agreement needs to be signed by an authorised agent of the company - the
place to check that would probably be psf at python.org (that's the email
address for the PSF board).

Assuming the subject line relates to the code that you would like to
contribute though, that particular change is unlikely to happen - 2.6 is
in maintenance mode and changing RLock from a Python implementation to
the faster C one is solidly in new feature territory. Although a
backport of the 3.2 C RLock implementation to 2.7 could be useful, I
doubt that backporting code provided by an existing committer would be
the subject of this query :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Thu Jan  7 14:11:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:11:27 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>

On Thu, Jan 7, 2010 at 13:23, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder.

Even if the contributed code as in this case is a method in an
existing file? How does that work, how do they keep ownership of one
method in the threading.py method? :-)

> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)

Ah. I probably misunderstood what the suggested contribution was.
Maybe it was a separate file, which I didn't get.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Thu Jan  7 14:15:09 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 07 Jan 2010 13:15:09 +0000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
References: <4B45AD8C.5000405@agama.tv>	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
Message-ID: <4B45DE5D.3030104@voidspace.org.uk>

On 07/01/2010 13:11, Lennart Regebro wrote:
> On Thu, Jan 7, 2010 at 13:23, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>    
>> As Simon pointed out, while some organisations do work that way, the PSF
>> isn't one of them.
>>
>> The PSF only requires that the code be contributed under a license that
>> then allows us to turn around and redistribute it under a different open
>> source license without requesting additional permission from the
>> copyright holder.
>>      
> Even if the contributed code as in this case is a method in an
> existing file? How does that work, how do they keep ownership of one
> method in the threading.py method? :-)
>
>    

When contributing code to Python all work remains copyright the original 
author. The combined work is copyright *everyone*. The PSF has a license 
to distribute it, which is all that is important.

How you meaningfully exercise your ownership over chunks of code is left 
for the reader to determine...

(i.e. copyright and ownership are legal terms that don't necessarily 
mean anything *practical* in these situations.)

Michael


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From stefan_ml at behnel.de  Thu Jan  7 14:30:16 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 14:30:16 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>
	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>
	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
Message-ID: <hi4nl7$2ej$1@ger.gmane.org>

Guido van Rossum, 07.01.2010 05:29:
> A better rule would be "you may access the memory buffer in a PyString
> or PyUnicode object with the GIL released as long as you own a
> reference to the string object." Everything else is out of bounds (or
> not worth the bother).

Is that a "yes" regarding the OP's original question about releasing the 
GIL during regexp searches?

Stefan



From regebro at gmail.com  Thu Jan  7 14:36:42 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 7 Jan 2010 14:36:42 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DE5D.3030104@voidspace.org.uk>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
	<319e029f1001070511i20d2aa4fw4a9e9b2f54fe84d8@mail.gmail.com>
	<4B45DE5D.3030104@voidspace.org.uk>
Message-ID: <319e029f1001070536t49f76899j111212b02c14f192@mail.gmail.com>

On Thu, Jan 7, 2010 at 14:15, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> (i.e. copyright and ownership are legal terms that don't necessarily mean
> anything *practical* in these situations.)

OK, fair enough. :-)
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Thu Jan  7 15:05:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Jan 2010 15:05:23 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <hi4pn2$9so$1@ger.gmane.org>

MRAB, 07.01.2010 04:07:
> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER

Py_UNICODE_TOLOWER looks safe to me at first glance.


> or PyErr_SetString?

Certainly not safe.


> Is there an easy way to find out?

Release it and fix any crashes? Note that this isn't a safe solution, 
though, as some GIL requiring code may be platform specific. So a better 
approach might be to extract any obviously problematic stuff from the 
existing code (such as any exception handling, explicit ref-counting or 
object creation), and *then* try to release the GIL.

Stefan



From solipsis at pitrou.net  Thu Jan  7 16:38:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 15:38:31 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <loom.20100107T163459-842@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> I know that it needs to have the GIL during memory-management calls, but
> does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there
> an easy way to find out?

There is no "easy way" to do so. The only safe way is to examine all the
functions or macros you want to call with the GIL released, and assess whether
it is safe to call them. As already pointed out, no reference count should be
changed, and generally no mutable container should be accessed, except if that
container is known not to be referenced anywhere else (that would be the case
for e.g. a list that your function has created and is busy populating).

I agree that releasing the GIL when doing non-trivial regex searches is a
worthwhile research, so please don't give up immediately :-)

Regards

Antoine Pitrou.




From olemis at gmail.com  Thu Jan  7 19:24:59 2010
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 7 Jan 2010 13:24:59 -0500
Subject: [Python-Dev] Question over splitting unittest into a package
In-Reply-To: <24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
References: <d80792120912300606m545d1d32x78d8c8cffc4cc762@mail.gmail.com>
	<1afaf6160912301204p247e77c4r2271c4f37114ba4@mail.gmail.com>
	<d80792120912310730u1676991eu6ea7e6d78a09821f@mail.gmail.com>
	<24ea26601001040624wb7d44ffl6ca0190aa33f8553@mail.gmail.com>
Message-ID: <24ea26601001071024m2abd988fuc77f94845d57483a@mail.gmail.com>

On Mon, Jan 4, 2010 at 9:24 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Dec 31, 2009 at 10:30 AM, Martin (gzlist) <gzlist at googlemail.com> wrote:
>> Thanks for the quick response.
>>
>> On 30/12/2009, Benjamin Peterson <benjamin at python.org> wrote:
>>
>> but maybe a
>> discussion could start about a new, less hacky, way of doing the same
>>
>
> I am strongly -1 for modifying the classes in ?traditional? unittest
> module [2]_ (except that I am strongly +1 for the package structure
> WITHOUT TOUCHING anything else ...) , and the more I think about it I
> am more convinced ... but anyway, this not a big deal (and in the end
> what I think is not that relevant either ... :o). So ...
>

IOW, if I had all the freedom to implement it, after the pkg structure
I'd do something like :

{{{
#!python

class TestResult:
    # Everything just the same
    def _is_relevant_tb_level(self, tb):
        return '__unittest' in tb.tb_frame.f_globals

class BetterTestResult(TestResult):
    # Further code ... maybe ;o)
    #
    def _is_relevant_tb_level(self, tb):
        # This or anything else you might want to do ;o)
        #
        globs = tb.tb_frame.f_globals
        is_relevant =  '__name__' in globs and \
            globs["__name__"].startswith("unittest")
        del globs
        return is_relevant
}}}

that's what inheritance is for ;o) ... but quite probably that's not
gonna happen, just a comment .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Ubuntu sustituye GIMP por F-Spot  -
http://feedproxy.google.com/~r/simelo-es/~3/-g48D6T6Ojs/ubuntu-sustituye-gimp-por-f-spot.html


From martin at v.loewis.de  Thu Jan  7 21:24:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:24:32 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi4nl7$2ej$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B45543C.2090200@gmail.com>	<1afaf6161001061932p9e7470esc6cdf7953b8c02fd@mail.gmail.com>	<ca471dc21001062029w6231077o9f728eea2b2d3427@mail.gmail.com>
	<hi4nl7$2ej$1@ger.gmane.org>
Message-ID: <4B464300.2020204@v.loewis.de>

>> A better rule would be "you may access the memory buffer in a PyString
>> or PyUnicode object with the GIL released as long as you own a
>> reference to the string object." Everything else is out of bounds (or
>> not worth the bother).
> 
> Is that a "yes" regarding the OP's original question about releasing the
> GIL during regexp searches?

No, because the regex engine may also operate on buffers that start
moving around when you release the GIL.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:27:09 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:27:09 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B46439D.9030604@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

I don't think that's possible. The regex engine can also operate on
objects whose representation may move in memory when you don't hold
the GIL (e.g. buffers that get mutated). Even if they stay in place -
if their contents changes, regex results may be confusing.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 21:31:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:31:21 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
Message-ID: <4B464499.4020305@v.loewis.de>

> I would like to use astgen.py to generate python classes corresponding to the 
> AST of something I have defined in a .asdl file, along the line of what is
> apparently done for the python AST itself. I thought astgen.py would
> take as an argument a .asdl file, but apparently it instead process a file
> called ast.txt. Where does this file come from ? Is it generated from
> Python.asdl ?

astgen.py is not used to process asdl files; ast.txt lives right next to
astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

HTH,
Martin


From foom at fuhm.net  Thu Jan  7 21:36:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 7 Jan 2010 15:36:37 -0500
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B46439D.9030604@v.loewis.de>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>

On Jan 7, 2010, at 3:27 PM, Martin v. L?wis wrote:

>> I've been wondering whether it's possible to release the GIL in the
>> regex engine during matching.
>
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated). Even if they stay in place -
> if their contents changes, regex results may be confusing.

It seems probably worthwhile to optimize for the common case of using  
the regexp engine on an immutable object of type "str" or "bytes", and  
allow releasing the GIL in *that* case, even if you have to keep it  
for the general case.

James

From martin at v.loewis.de  Thu Jan  7 21:45:42 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:45:42 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<82C03488-5D03-4E67-8B67-CE6EE5879D2B@fuhm.net>
Message-ID: <4B4647F6.2060309@v.loewis.de>

>>> I've been wondering whether it's possible to release the GIL in the
>>> regex engine during matching.
>>
>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated). Even if they stay in place -
>> if their contents changes, regex results may be confusing.
> 
> It seems probably worthwhile to optimize for the common case of using
> the regexp engine on an immutable object of type "str" or "bytes", and
> allow releasing the GIL in *that* case, even if you have to keep it for
> the general case.

Right. This problem was the one that I thought of first.

Thinking about these things is fairly difficult (to me, at least), so
I think I could only tell whether I would consider a patch thread-safe
that released the GIL around matching under selected circumstances -
if I had the patch available. I don't see any obvious reason (assuming
Guido's list of conditions holds - i.e. you are holding references to
everything you access).

Regards,
Martin


From ncoghlan at gmail.com  Thu Jan  7 21:48:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 08 Jan 2010 06:48:05 +1000
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45DECB.7070907@agama.tv>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com> <4B45DECB.7070907@agama.tv>
Message-ID: <4B464885.40806@gmail.com>

Johan Gill wrote:
> Yes, it is the new RLock implementation.
> If I understood this correctly, we should make a patch against trunk if
> anything should be contributed.

Yep.

> Do you mean that we wouldn't need the paperwork for backporting the
> original patch committed to py3k?

Whether or not a contributor agreement was essential for this particular
contribution would depend on how much new code was needed for the
backport, but the bulk of the copyright on the C RLock code would remain
with Antoine regardless.

However, sorting through the legalities of the contributor agreement
really is the best way to make sure every is squared away nice and
neatly from a legal point of view.

After all, even if I was a lawyer (which I'm not, I'm just a developer
with an interest in licensing issues), I still wouldn't be *your* lawyer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Jan  7 21:43:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 7 Jan 2010 20:43:17 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?GIL_required_for_=5Fall=5F_Python_calls=3F?=
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
Message-ID: <loom.20100107T214211-130@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I don't think that's possible. The regex engine can also operate on
> objects whose representation may move in memory when you don't hold
> the GIL (e.g. buffers that get mutated).

Why is it a problem? If we get a buffer through the new buffer API, the object
should ensure that the representation isn't moved away until the buffer is 
released.

Regards

Antoine.




From martin at v.loewis.de  Thu Jan  7 21:59:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 21:59:29 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <4B45500C.8090905@mrabarnett.plus.com>
References: <4B45500C.8090905@mrabarnett.plus.com>
Message-ID: <4B464B31.7040406@v.loewis.de>

> I've been wondering whether it's possible to release the GIL in the
> regex engine during matching.

Ok, here is another problem: SRE_OP_REPEAT uses PyObject_MALLOC,
which requires the GIL (it then also may call PyErr_NoMemory,
which also requires the GIL).

Regards,
Martin


From johan.gill at agama.tv  Thu Jan  7 14:16:59 2010
From: johan.gill at agama.tv (Johan Gill)
Date: Thu, 07 Jan 2010 14:16:59 +0100
Subject: [Python-Dev] Backported faster RLock to Python 2.6.
In-Reply-To: <4B45D22F.40404@gmail.com>
References: <4B45AD8C.5000405@agama.tv>
	<319e029f1001070312q136bd933u669c1291fcb1b602@mail.gmail.com>
	<4B45D22F.40404@gmail.com>
Message-ID: <4B45DECB.7070907@agama.tv>

On 01/07/2010 01:23 PM, Nick Coghlan wrote:
> As Simon pointed out, while some organisations do work that way, the PSF
> isn't one of them.
>
> The PSF only requires that the code be contributed under a license that
> then allows us to turn around and redistribute it under a different open
> source license without requesting additional permission from the
> copyright holder. For corporate contributions, I believe the contributor
> agreement needs to be signed by an authorised agent of the company - the
> place to check that would probably be psf at python.org (that's the email
> address for the PSF board).
>
> Assuming the subject line relates to the code that you would like to
> contribute though, that particular change is unlikely to happen - 2.6 is
> in maintenance mode and changing RLock from a Python implementation to
> the faster C one is solidly in new feature territory. Although a
> backport of the 3.2 C RLock implementation to 2.7 could be useful, I
> doubt that backporting code provided by an existing committer would be
> the subject of this query :)
>
> Regards,
> Nick.
>
>    
Yes, it is the new RLock implementation.
If I understood this correctly, we should make a patch against trunk if 
anything should be contributed.
Do you mean that we wouldn't need the paperwork for backporting the 
original patch committed to py3k?

Regards
Johan Gill
Agama Technologies



From yoann.padioleau at facebook.com  Thu Jan  7 22:07:55 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Thu, 7 Jan 2010 13:07:55 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464499.4020305@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
Message-ID: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>


On Jan 7, 2010, at 12:31 PM, Martin v. L?wis wrote:

>> I would like to use astgen.py to generate python classes corresponding to the 
>> AST of something I have defined in a .asdl file, along the line of what is
>> apparently done for the python AST itself. I thought astgen.py would
>> take as an argument a .asdl file, but apparently it instead process a file
>> called ast.txt. Where does this file come from ? Is it generated from
>> Python.asdl ?
> 
> astgen.py is not used to process asdl files; ast.txt lives right next to
> astgen.py. Instead, the asdl file is processed by Parser/asdl_c.py.

Yes, I know that. That's why I asked about the relation between ast.txt and Python.adsl.
If internally the parser uses the .adsl, but expose as a reflection mechanism things
that were generated from ast.txt, then there could be a mismatch. Where does ast.txt comes from ? Shouldn't it be generated itself from Python.adsl ?

So we would have

Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py containing all the UnarySub, Expression, classes that represents a Python AST.



> 
> HTH,
> Martin



From martin at v.loewis.de  Thu Jan  7 22:11:36 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 07 Jan 2010 22:11:36 +0100
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <loom.20100107T214211-130@post.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
Message-ID: <4B464E08.5020703@v.loewis.de>

>> I don't think that's possible. The regex engine can also operate on
>> objects whose representation may move in memory when you don't hold
>> the GIL (e.g. buffers that get mutated).
> 
> Why is it a problem? If we get a buffer through the new buffer API, the object
> should ensure that the representation isn't moved away until the buffer is 
> released.

In 2.7, we currently get the buffer with bf_getreadbuffer. In 3.x, we have

    /* Release the buffer immediately --- possibly dangerous
       but doing something else would require some re-factoring
    */
    PyBuffer_Release(&view);


Even if we do use the new API, and correctly, it still might be
confusing if the contents of the buffer changes underneath.

Regards,
Martin


From martin at v.loewis.de  Thu Jan  7 22:16:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 07 Jan 2010 22:16:12 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
Message-ID: <4B464F1C.7020404@v.loewis.de>

>> astgen.py is not used to process asdl files; ast.txt lives right
>> next to astgen.py. Instead, the asdl file is processed by
>> Parser/asdl_c.py.
> 
> Yes, I know that. That's why I asked about the relation between
> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
> expose as a reflection mechanism things that were generated from
> ast.txt, then there could be a mismatch. Where does ast.txt comes
> from ? Shouldn't it be generated itself from Python.adsl ?

What you may not be aware of is that Tools/compiler (and the
compiler package that it builds on) are both unused and unmaintained.

If the package stops working correctly - tough luck.

> So we would have
> 
> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
> containing all the UnarySub, Expression, classes that represents a
> Python AST.

No - what actually happens in Python 3.x is this: both the compiler
package and Tools/compiler are removed.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 01:10:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 01:10:35 +0100
Subject: [Python-Dev] Improve open() to support reading file starting with
	an unicode BOM
Message-ID: <201001080110.35800.victor.stinner@haypocalc.com>

Hi,

Builtin open() function is unable to open an UTF-16/32 file starting with a 
BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
file starting with a BOM, read()/readline() returns also the BOM whereas the 
BOM should be "ignored".

See recent issues related to reading an UTF-8 text file including a BOM: #7185 
(csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with 
the UTF-8-SIG encoding, but it's possible to do better.

I propose to improve open() (TextIOWrapper) by using the BOM to choose the 
right encoding. I think that only files opened in read only mode should 
support this new feature. *Read* the BOM in a *write* only file would cause 
unexpected behaviours.

Since my proposition changes the result TextIOWrapper.read()/readline() for 
files starting with a BOM, we might introduce an option to open() to enable 
the new behaviour. But is it really needed to keep the backward compatibility?

I wrote a proof of concept attached to the issue #7651. My patch only changes 
the behaviour of TextIOWrapper for reading files starting with a BOM. It 
doesn't work yet if a seek() is used before the first read.

-- 
Victor Stinner
http://www.haypocalc.com/


From guido at python.org  Fri Jan  8 01:52:20 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 16:52:20 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>

I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
talk. And for the other two, perhaps it would make more sense to have
a separate encoding-guessing function that takes a binary stream and
returns a text stream wrapping it with the proper encoding?

--Guido

On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
> file starting with a BOM, read()/readline() returns also the BOM whereas the
> BOM should be "ignored".
>
> See recent issues related to reading an UTF-8 text file including a BOM: #7185
> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
> the UTF-8-SIG encoding, but it's possible to do better.
>
> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
> right encoding. I think that only files opened in read only mode should
> support this new feature. *Read* the BOM in a *write* only file would cause
> unexpected behaviours.
>
> Since my proposition changes the result TextIOWrapper.read()/readline() for
> files starting with a BOM, we might introduce an option to open() to enable
> the new behaviour. But is it really needed to keep the backward compatibility?
>
> I wrote a proof of concept attached to the issue #7651. My patch only changes
> the behaviour of TextIOWrapper for reading files starting with a BOM. It
> doesn't work yet if a seek() is used before the first read.
>
> --
> Victor Stinner
> http://www.haypocalc.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From python at mrabarnett.plus.com  Fri Jan  8 03:23:08 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 02:23:08 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <4B46970C.9010306@mrabarnett.plus.com>

Guido van Rossum wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
> 
Alternatively, have a universal UTF-8/16/32 encoding, ie one that 
expects UTF-8,
with or without BOM, or UTF-16/32 with BOM.
> 
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
>> See recent issues related to reading an UTF-8 text file including a BOM: #7185
>> (csv) and #7519 (ConfigParser). Such file can be opened in unicode mode with
>> the UTF-8-SIG encoding, but it's possible to do better.
>>
>> I propose to improve open() (TextIOWrapper) by using the BOM to choose the
>> right encoding. I think that only files opened in read only mode should
>> support this new feature. *Read* the BOM in a *write* only file would cause
>> unexpected behaviours.
>>
>> Since my proposition changes the result TextIOWrapper.read()/readline() for
>> files starting with a BOM, we might introduce an option to open() to enable
>> the new behaviour. But is it really needed to keep the backward compatibility?
>>
>> I wrote a proof of concept attached to the issue #7651. My patch only changes
>> the behaviour of TextIOWrapper for reading files starting with a BOM. It
>> doesn't work yet if a seek() is used before the first read.
>>


From nick.bastin at gmail.com  Fri Jan  8 04:11:03 2010
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Thu, 7 Jan 2010 22:11:03 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
Message-ID: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>

I think this problem probably needs to move over to distutils-sig, as
it doesn't seem to be specific to the way that Python itself uses
distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
linux and solaris and automatically adds '.' to the library_dirs, and
I suspect it just needs to do this on FreeBSD as well (adding bsd to
the list of platforms for which this is performed "solves" the
problem, but I don't pretend to know enough about either distutils or
freebsd to determine if this is the correct solution).

--
Nick


From glyph at twistedmatrix.com  Fri Jan  8 04:34:36 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Thu, 7 Jan 2010 22:34:36 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>



On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>> 
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".

> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

It *is* crazy, but unfortunately rather common.  Wikipedia has a good description of the issues: <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as being UTF-8, so it's become a convention to do that.  That's not good enough, so you need to guess the encoding as well to make sure, but if there is a BOM and you can otherwise verify that the file is probably UTF-8 encoded, you should discard it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100107/1bc40870/attachment-0007.htm>

From tseaver at palladion.com  Fri Jan  8 05:17:28 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 07 Jan 2010 23:17:28 -0500
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <hi6bko$d0h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.  distutils.command.build_ext tests for Py_ENABLE_SHARED on
> linux and solaris and automatically adds '.' to the library_dirs, and
> I suspect it just needs to do this on FreeBSD as well (adding bsd to
> the list of platforms for which this is performed "solves" the
> problem, but I don't pretend to know enough about either distutils or
> freebsd to determine if this is the correct solution).

I wouldn't say it needed discussion on the SIG:  just create a bug
report, with the tentative patch you have worked out, and get it
assigned to Tarek.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGsdQACgkQ+gerLs4ltQ5BMQCgtV8snMXH/6dDwgdN4sIJljLd
koYAoKq6c0tKsRSrITHcygu4Od9FVzF5
=BJaE
-----END PGP SIGNATURE-----



From guido at python.org  Fri Jan  8 05:21:04 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 7 Jan 2010 20:21:04 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
Message-ID: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>

On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>
>
> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>
> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>
> Hi,
>
> Builtin open() function is unable to open an UTF-16/32 file starting with a
>
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>
> file starting with a BOM, read()/readline() returns also the BOM whereas the
>
> BOM should be "ignored".
>
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>
> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
> description of the issues:
> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
> being UTF-8, so it's become a convention to do that. ?That's not good
> enough, so you need to guess the encoding as well to make sure, but if there
> is a BOM and you can otherwise verify that the file is probably UTF-8
> encoded, you should discard it.

That doesn't make sense. If the file isn't UTF-8 you can't see the
BOM, because the BOM itself is UTF-8-encoded.

(And yes, I know this happens. Doesn't mean we need to auto-guess by
default; there are lots of issues e.g. what should happen after
seeking to offset 0?)

-- 
--Guido van Rossum (python.org/~guido)


From stephen at xemacs.org  Fri Jan  8 07:06:16 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 08 Jan 2010 15:06:16 +0900
Subject: [Python-Dev] Improve open() to support reading file
	starting	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
 > talk.

That doesn't stop many applications from doing it.  Python should
perhaps<wink,nudge> not produce UTF-8 + BOM without a disclaimer of
indemnification against all resulting damage, signed in blood, from
the user for each instance.

But it should do something sane when reading such files.  I can't
really see any harm in throwing it away, especially since use of
ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
IIRC.






From tseaver at palladion.com  Fri Jan  8 07:12:12 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 01:12:12 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <hi6ibr$qag$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>
>> BOM should be "ignored".
>>
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.
> 
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

The BOM should not be seekeable if the file is opened with the proposed
"guess encoding from BOM" mode:  it isn't properly part of the stream at
all in that case.

A UTF-8 BOM is an absurditiy, but it exists *everywhere* in the wild:
Python would do wll to make it as easy as possible to consume such
files, as well as the non-insane versions (UTF-16 / UTF-32 BOMs).  In
the best of all possible worlds, I would just try opening the file so:

  f = open('/path/to/file', 'r', encoding="DWIFM")

and any BOM present would set the encoding for the remainder of the stream..



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktGzLsACgkQ+gerLs4ltQ5+cwCdGfycPdj6+cPfD23vH644SpHL
sI0AoLGD7nfgMEJdJhBr90yjQQHfDgcJ
=js+2
-----END PGP SIGNATURE-----



From glyph at twistedmatrix.com  Fri Jan  8 08:55:27 2010
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 8 Jan 2010 02:55:27 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>


On Jan 7, 2010, at 11:21 PM, Guido van Rossum wrote:

> On Thu, Jan 7, 2010 at 7:34 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> 
>> On Jan 7, 2010, at 7:52 PM, Guido van Rossum wrote:
>>> 
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>> 
>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.  If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal.  It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I think it's pretty clear that the BOM should still be skipped in that case ...



From martin at v.loewis.de  Fri Jan  8 10:05:17 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:05:17 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <4B46F54D.9090103@v.loewis.de>

>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>> description of the issues:
>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>> being UTF-8, so it's become a convention to do that.  That's not good
>> enough, so you need to guess the encoding as well to make sure, but if there
>> is a BOM and you can otherwise verify that the file is probably UTF-8
>> encoded, you should discard it.
> 
> That doesn't make sense. If the file isn't UTF-8 you can't see the
> BOM, because the BOM itself is UTF-8-encoded.

I think what Glyph meant is this: if a file starts with the UTF-8
signature, assume it's UTF-8. Then validate the assumption against the
rest of the file also, and then process it as UTF-8. If the rest clearly
is not UTF-8, assume that the UTF-8 signature is bogus.

I understood this proposal as a general processing guideline, not
something the io library should do (but, say, a text editor).

FWIW, I'm personally in favor of using the UTF-8 signature. If people
consider them crazy talk, that may be because UTF-8 can't possibly have
a byte order - hence I call it a signature, not the BOM. As a signature,
I don't consider it crazy at all. There is a long tradition of having
magic bytes in files (executable files, Postscript, PDF, ... - see
/etc/magic). Having a magic byte sequence for plain text to denote the
encoding is useful and helps reducing moji-bake. This is the reason it's
used on Windows: notepad would normally assume that text is in the ANSI
code page, and for compatibility, it can't stop doing that. So the UTF-8
signature gives them an exit strategy.

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:06:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:06:34 +0100
Subject: [Python-Dev] Improve open() to support reading file	starting
 with an unicode BOM
In-Reply-To: <87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<87fx6hjfl3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4B46F59A.5060905@v.loewis.de>

> But it should do something sane when reading such files.  I can't
> really see any harm in throwing it away, especially since use of
> ZERO-WIDTH NO-BREAK SPACE as a joining character has been deprecated
> IIRC.

And indeed it does, when you open the file in the utf-8-sig encoding.

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 10:08:30 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 10:08:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46970C.9010306@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<4B46970C.9010306@mrabarnett.plus.com>
Message-ID: <201001081008.30904.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 03:23:08, MRAB a ?crit :
> Guido van Rossum wrote:
> > I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> > talk. And for the other two, perhaps it would make more sense to have
> > a separate encoding-guessing function that takes a binary stream and
> > returns a text stream wrapping it with the proper encoding?
> 
> Alternatively, have a universal UTF-8/16/32 encoding, ie one that
> expects UTF-8,
> with or without BOM, or UTF-16/32 with BOM.

Do you mean open(filename, encoding="BOM")? I suppose that "BOM" would be a 
magical value specific to read a text file (open(filename, "r")), not a real 
codec?

Otherwise which encoding should be used for open(filename, "w", 
encoding="BOM")?

-- 
Victor Stinner
http://www.haypocalc.com/


From martin at v.loewis.de  Fri Jan  8 10:10:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:10:23 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with	an unicode BOM
In-Reply-To: <201001080110.35800.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
Message-ID: <4B46F67F.60604@v.loewis.de>

> Builtin open() function is unable to open an UTF-16/32 file starting with a 
> BOM if the encoding is not specified (raise an unicode error). For an UTF-8 
> file starting with a BOM, read()/readline() returns also the BOM whereas the 
> BOM should be "ignored".

It depends. If you use the utf-8-sig encoding, it *will* ignore the
UTF-8 signature.

> Since my proposition changes the result TextIOWrapper.read()/readline() for 
> files starting with a BOM, we might introduce an option to open() to enable 
> the new behaviour. But is it really needed to keep the backward compatibility?

Absolutely. And there is no need to produce a new option, but instead
use the existing options: define an encoding that auto-detects the
encoding from the family of BOMs. Maybe you call it encoding="sniff".

Regards,
Martin


From martin at v.loewis.de  Fri Jan  8 10:11:51 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 08 Jan 2010 10:11:51 +0100
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>	<4B450CF8.8090009@v.loewis.de>	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
Message-ID: <4B46F6D7.1050301@v.loewis.de>

Nicholas Bastin wrote:
> I think this problem probably needs to move over to distutils-sig, as
> it doesn't seem to be specific to the way that Python itself uses
> distutils.

I'm fairly skeptical that anybody on distutils SIG is interested in
details of the Python build process...

Regards,
Martin


From victor.stinner at haypocalc.com  Fri Jan  8 11:27:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:27:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
Message-ID: <201001081127.44044.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
(...)
> (And yes, I know this happens. Doesn't mean we need to auto-guess by
> default; there are lots of issues e.g. what should happen after
> seeking to offset 0?)

I wrote a new version of my patch (version 3):

 * don't change the default behaviour: use open(filename, encoding="BOM") to 
check the BOM is there is any
 * fix for seek(0): always ignore the BOM
 * add an unit test: check that the right encoding is detect, but also the the 
BOM is ignored (especially after a seek(0))

BOM encoding doesn't work for writing into a file, so open(filename, "w", 
encoding="BOM") raises a ValueError.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:31:37 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:31:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <201001081131.37492.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 01:52:20, Guido van Rossum a ?crit :
> And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?

I choosed to modify open()+TextIOWrapper instead of writing a new function 
because I would like to avoid an extra read operation (syscall) on the file. 
With my implementation, no specific read operation is needed to detect the 
BOM. The BOM is simply checked in the first _read_chunk().

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Fri Jan  8 11:40:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 11:40:28 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Improve_open=28=29_to_support_reading?=
 =?iso-8859-1?q?_file_starting_with=09an_unicode_BOM?=
In-Reply-To: <4B46F67F.60604@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
Message-ID: <201001081140.28124.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
> > Builtin open() function is unable to open an UTF-16/32 file starting with
> > a BOM if the encoding is not specified (raise an unicode error). For an
> > UTF-8 file starting with a BOM, read()/readline() returns also the BOM
> > whereas the BOM should be "ignored".
> 
> It depends. If you use the utf-8-sig encoding, it *will* ignore the
> UTF-8 signature.

Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
remove the BOM after the first read (much harder if you use a module like 
ConfigParser or csv).

> > Since my proposition changes the result TextIOWrapper.read()/readline()
> > for files starting with a BOM, we might introduce an option to open() to
> > enable the new behaviour. But is it really needed to keep the backward
> > compatibility?
> 
> Absolutely. And there is no need to produce a new option, but instead
> use the existing options: define an encoding that auto-detects the
> encoding from the family of BOMs. Maybe you call it encoding="sniff".

Good idea, I choosed open(filename, encoding="BOM").

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Fri Jan  8 15:27:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:27:42 +0000 (UTC)
Subject: [Python-Dev] GIL required for _all_ Python calls?
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de>
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de>
Message-ID: <hi7fcu$gs7$1@ger.gmane.org>

Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
> 
> Even if we do use the new API, and correctly, it still might be
> confusing if the contents of the buffer changes underneath.

Well, no more confusing than when you compute a SHA1 hash or zlib-
compress the buffer, is it?

Regards

Antoine




From solipsis at pitrou.net  Fri Jan  8 15:34:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 14:34:26 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <loom.20100108T153305-228@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any

Well, I think if we implement this the default behaviour *should* be changed.
It looks a bit senseless to have two different "auto-choose" options, one with
encoding=None and one with encoding="BOM".

Regards

Antoine.




From guido at python.org  Fri Jan  8 16:48:44 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:48:44 -0800
Subject: [Python-Dev] GIL required for _all_ Python calls?
In-Reply-To: <hi7fcu$gs7$1@ger.gmane.org>
References: <4B45500C.8090905@mrabarnett.plus.com>
	<4B46439D.9030604@v.loewis.de> 
	<loom.20100107T214211-130@post.gmane.org>
	<4B464E08.5020703@v.loewis.de> <hi7fcu$gs7$1@ger.gmane.org>
Message-ID: <ca471dc21001080748lbc18bbx4c4ec2d3f4f027e@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Thu, 07 Jan 2010 22:11:36 +0100, Martin v. L?wis a ?crit?:
>>
>> Even if we do use the new API, and correctly, it still might be
>> confusing if the contents of the buffer changes underneath.
>
> Well, no more confusing than when you compute a SHA1 hash or zlib-
> compress the buffer, is it?

That depends. Algorithms that make exactly one pass over the buffer
will run fine (maybe producing a meaningless result). But the regex
matcher may scan the buffer repeatedly (for backtracking purposes) and
it would take a considerable analysis to prove that cannot mess up its
internal data structures if the data underneath changes. (I give it a
decent chance that it's fine, but since it was written without ever
considering this possibility I'm not 100% sure.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:52:48 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:52:48 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<B3F81F02-1DFF-464B-8425-F32CC3D84EA7@twistedmatrix.com>
Message-ID: <ca471dc21001080752r312dd47fx32486a95336160ec@mail.gmail.com>

On Thu, Jan 7, 2010 at 11:55 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
> I'm saying that the BOM itself isn't enough to detect that the file is actually UTF-8.

And I'm saying that it is, with as much certainty as we can ever guess
the encoding of a file.

> If (for whatever reason: explicitly specified, guessed in some other way) the file's encoding is determined to be something else, the bytes comprising the BOM should be decoded as normal. ?It's just that the UTF-8 decoding of the BOM at the start of a file should be "".

Sure, a Latin-1-encoded file could start with the same pattern that is
a UTF-8-encoded BOM. But at that point, a UTF-16-encoded file is also
valid Latin-1.

The question was in the context of encoding-guessing; if we're
guessing, a UTF-8-encoded BOM cannot signify anything else but UTF-8.
(Ditto for UTF-16 and UTF-32 BOMs.)

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:54:15 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:54:15 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100108T153305-228@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
Message-ID: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>

On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>
>> I wrote a new version of my patch (version 3):
>>
>> ?* don't change the default behaviour: use open(filename, encoding="BOM") to
>> check the BOM is there is any
>
> Well, I think if we implement this the default behaviour *should* be changed.
> It looks a bit senseless to have two different "auto-choose" options, one with
> encoding=None and one with encoding="BOM".

Well there *are* two different auto options: use the environment
variables (LANG etc.) or inspect the contents of the file. I think it
would be useful to have ways to specify both.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:56:46 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:56:46 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <ca471dc21001080756p5cb9f560x2a4443efa441fa7b@mail.gmail.com>

On Fri, Jan 8, 2010 at 1:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It *is* crazy, but unfortunately rather common. ?Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>. ?Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that. ?That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>>
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
>
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.
>
> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
>
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Sure. I said "crazy talk" only to stir up discussion. Which worked. :-)

Also, I don't want Python's default behavior to change -- sniffing the
encoding should be a separate option.

-- 
--Guido van Rossum (python.org/~guido)


From guido at python.org  Fri Jan  8 16:59:45 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 8 Jan 2010 07:59:45 -0800
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi6ibr$qag$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com> 
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com> 
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com> 
	<hi6ibr$qag$1@ger.gmane.org>
Message-ID: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>

On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
> The BOM should not be seekeable if the file is opened with the proposed
> "guess encoding from BOM" mode: ?it isn't properly part of the stream at
> all in that case.

This feels about right to me. There are still questions though:
immediately after opening a file with a BOM, what should .tell()
return? And regardless of that, .seek(0) should put the file in that
same initial state.

-- 
--Guido van Rossum (python.org/~guido)


From solipsis at pitrou.net  Fri Jan  8 17:03:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:03:07 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <loom.20100108T170053-283@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> > Well, I think if we implement this the default behaviour *should* be changed.
> > It looks a bit senseless to have two different "auto-choose" options, one
with
> > encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Yes, perhaps. In the context of open() however I think it would be helpful to
change the default.
Moreover, reading the BOM is certainly much more reliable than our current
guessing based on the locale or the "device encoding".

Regards

Antoine.




From mal at egenix.com  Fri Jan  8 17:25:22 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 17:25:22 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
Message-ID: <4B475C72.1010207@egenix.com>

Guido van Rossum wrote:
> On Fri, Jan 8, 2010 at 6:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>>
>>> I wrote a new version of my patch (version 3):
>>>
>>>  * don't change the default behaviour: use open(filename, encoding="BOM") to
>>> check the BOM is there is any
>>
>> Well, I think if we implement this the default behaviour *should* be changed.
>> It looks a bit senseless to have two different "auto-choose" options, one with
>> encoding=None and one with encoding="BOM".
> 
> Well there *are* two different auto options: use the environment
> variables (LANG etc.) or inspect the contents of the file. I think it
> would be useful to have ways to specify both.

Shouldn't this encoding guessing be a separate function that you call
on either a file or a seekable stream ?

After all, detecting encodings is just as useful to have for non-file
streams. You'd then avoid having to stuff everything into
a single function call and also open up the door for more complex
application specific guess work or defaults.

The whole process would then have two steps:

 1. guess encoding

  import codecs
  encoding = codecs.guess_file_encoding(filename)

 2. open the file with the found encoding

  f = open(filename, encoding=encoding)

For seekable streams f, you'd have:

 1. guess encoding

  import codecs
  encoding = codecs.guess_stream_encoding(f)

 2. wrap the stream with a reader for the found encoding

  reader_class = codecs.getreader(encoding)
  g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From solipsis at pitrou.net  Fri Jan  8 17:31:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 8 Jan 2010 16:31:33 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Improve_open=28=29_to_support_reading_file?=
	=?utf-8?q?_starting=09with_an_unicode_BOM?=
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <loom.20100108T171644-311@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver <at> palladion.com>
wrote:
> > The BOM should not be seekeable if the file is opened with the proposed
> > "guess encoding from BOM" mode:  it isn't properly part of the stream at
> > all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return?

tell() in the context of text I/O is specified to return an "opaque cookie". So
whatever value it returns would probably be fine, as long as seeking to that
value leaves the file in an acceptable state.

Rewinding (seeking to 0) in the presence of a BOM is already reasonably
supported by the TextIOWrapper object:

>>> dec = codecs.getincrementaldecoder('utf-16')()
>>> dec.decode(b'\xff\xfea\x00b\x00')
'ab'
>>> dec.decode(b'\xff\xfea\x00b\x00')
'\ufeffab'
>>> 
>>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
>>> f = io.TextIOWrapper(bio, encoding='utf-16')
>>> f.read()
'ab'
>>> f.seek(0)
0
>>> f.read()
'ab'

There are tests for this in test_io.py (test_encoded_writes, line 1929, and
test_append_bom and test_seek_bom, line 2045).

Regards

Antoine.




From python at mrabarnett.plus.com  Fri Jan  8 17:47:18 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 Jan 2010 16:47:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081127.44044.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
Message-ID: <4B476196.4080409@mrabarnett.plus.com>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 05:21:04, Guido van Rossum a ?crit :
> (...)
>> (And yes, I know this happens. Doesn't mean we need to auto-guess by
>> default; there are lots of issues e.g. what should happen after
>> seeking to offset 0?)
> 
> I wrote a new version of my patch (version 3):
> 
>  * don't change the default behaviour: use open(filename, encoding="BOM") to 
> check the BOM is there is any
>  * fix for seek(0): always ignore the BOM
>  * add an unit test: check that the right encoding is detect, but also the the 
> BOM is ignored (especially after a seek(0))
> 
> BOM encoding doesn't work for writing into a file, so open(filename, "w", 
> encoding="BOM") raises a ValueError.
> 
I think it's similar to universal newline mode. You can tell it that
you're reading UTF-something-encoded text (common forms only).

The preference is UTF-8, but it could be UTF-8-sig (from Windows), or
possibly UTF-16/32, which really need a BOM because there are multiple
bytes per codepoint, so the bytes could be big-endian or little-endian.

The BOM (or signature) tells you what the encoding is, defaulting to
UTF-8 if there's none. If it subsequently raises a DecodeError, then
so be it!

Maybe there should also be a way of determining what encoding it decided
it was, so that you can then write a new file in that same encoding.


From status at bugs.python.org  Fri Jan  8 18:07:13 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Jan 2010 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100108170714.014B078669@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/01/10 - 01/08/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2544 open (+27) / 16937 closed (+15) / 19481 total (+42)

Open issues with patches:  1017

Average duration of open issues: 708 days.
Median duration of open issues: 464 days.

Open Issues Breakdown
   open  2509 (+27)
pending    34 ( +0)

Issues Created Or Reopened (43)
_______________________________

Extended slicing with classic class behaves strangely            01/07/10
       http://bugs.python.org/issue7532    reopened mark.dickinson                
       patch                                                                   

optparse library documentation has an insignificant formatting i 01/01/10
CLOSED http://bugs.python.org/issue7618    created  vazovsky                      
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6           01/01/10
CLOSED http://bugs.python.org/issue7619    created  djc                           
                                                                               

Vim syntax highlight                                             01/02/10
       http://bugs.python.org/issue7620    created  july                          
       patch                                                                   

Test issue                                                       01/02/10
CLOSED http://bugs.python.org/issue7621    created  georg.brandl                  
                                                                               

[patch] improve unicode methods: split() rsplit() and replace()  01/03/10
       http://bugs.python.org/issue7622    created  flox                          
       patch                                                                   

PropertyType missing in Lib/types.py                             01/03/10
CLOSED http://bugs.python.org/issue7623    created  wplappert                     
                                                                               

isinstance(... ,collections.Callable) fails with oldstyle class  01/03/10
       http://bugs.python.org/issue7624    created  rgammans                      
                                                                               

bytearray needs more tests for "b.some_method()[0] is not b"     01/03/10
       http://bugs.python.org/issue7625    created  flox                          
       patch                                                                   

Entity references without semicolon in HTMLParser                01/03/10
CLOSED http://bugs.python.org/issue7626    created  stefan.schweizer              
                                                                               

mailbox.MH.remove() lock handling is broken                      01/04/10
       http://bugs.python.org/issue7627    created  sraustein                     
                                                                               

round() doesn't work correctly in 3.1.1                          01/04/10
CLOSED http://bugs.python.org/issue7628    created  bkovt                         
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs" 01/04/10
CLOSED http://bugs.python.org/issue7629    created  popelkopp                     
                                                                               

Strange behaviour of decimal.Decimal                             01/04/10
CLOSED http://bugs.python.org/issue7630    created  parmax                        
                                                                               

undefined label: bltin-file-objects                              01/04/10
CLOSED http://bugs.python.org/issue7631    created  ezio.melotti                  
                                                                               

dtoa.c: oversize b in quorem                                     01/04/10
       http://bugs.python.org/issue7632    created  skrah                         
                                                                               

decimal.py: type conversion in context methods                   01/04/10
       http://bugs.python.org/issue7633    created  skrah                         
       patch, easy                                                             

next/previous links in documentation skip some sections          01/05/10
CLOSED http://bugs.python.org/issue7634    created  gagenellina                   
                                                                               

19.6 xml.dom.pulldom doc: stub?                                  01/05/10
       http://bugs.python.org/issue7635    created  tjreedy                       
                                                                               

Add a set update action to optparse                              01/05/10
CLOSED http://bugs.python.org/issue7636    created  hardkrash                     
       patch                                                                   

Improve 19.5. xml.dom.minidom doc                                01/05/10
       http://bugs.python.org/issue7637    created  tjreedy                       
                                                                               

Counterintuitive str.splitlines() inconsistency.                 01/05/10
CLOSED http://bugs.python.org/issue7638    created  vencabot_teppoo               
                                                                               

bdist_msi fails on files with long names                         01/05/10
       http://bugs.python.org/issue7639    created  mmm77                         
                                                                               

buffered io seek() buggy                                         01/05/10
       http://bugs.python.org/issue7640    created  pakal                         
       patch                                                                   

Built-in Formatter accepts undocumented presentation type        01/06/10
CLOSED http://bugs.python.org/issue7641    created  lrekucki                      
                                                                               

[patch] Minor improvement in os.system doc                       01/06/10
       http://bugs.python.org/issue7642    created  Val                           
       patch                                                                   

What is an ASCII linebreak?                                      01/06/10
       http://bugs.python.org/issue7643    created  flox                          
                                                                               

bug in nntplib.body() method with possible fix                   01/06/10
       http://bugs.python.org/issue7644    created  mdmullins                     
       easy                                                                    

test_distutils fails on Windows XP                               01/06/10
       http://bugs.python.org/issue7645    created  austin987                     
                                                                               

test_telnetlib fails in Windows XP                               01/06/10
       http://bugs.python.org/issue7646    created  austin987                     
                                                                               

Add statvfs flags to the posix module                            01/06/10
       http://bugs.python.org/issue7647    created  ajax at redhat.com               
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                 01/06/10
       http://bugs.python.org/issue7648    created  austin987                     
       easy                                                                    

"u'%c' % char" broken for chars in range '\x80'-'\xFF'           01/07/10
       http://bugs.python.org/issue7649    created  ezio.melotti                  
       patch, easy                                                             

test_uuid is invalid                                             01/07/10
       http://bugs.python.org/issue7650    created  austin987                     
                                                                               

Python3: guess text file charset using the BOM                   01/07/10
       http://bugs.python.org/issue7651    created  haypo                         
       patch                                                                   

Merge C version of decimal into py3k.                            01/07/10
       http://bugs.python.org/issue7652    created  mark.dickinson                
                                                                               

Fix description of the way the PythonPath Windows registry key w 01/07/10
CLOSED http://bugs.python.org/issue7653    created  BoarGules                     
       patch, needs review                                                     

[patch] Enable additional bytes and memoryview tests.            01/07/10
       http://bugs.python.org/issue7654    created  flox                          
       patch                                                                   

PEP 374 Title usage & script redirection typo fixes              01/07/10
CLOSED http://bugs.python.org/issue7655    created  bab9e9                        
       patch                                                                   

test_hashlib fails on some installations (specifically Neal's re 01/08/10
       http://bugs.python.org/issue7656    created  r.david.murray                
                                                                               

test_ctypes failure on AIX 5.3                                   01/08/10
       http://bugs.python.org/issue7657    created  mallayya                      
       patch                                                                   

OS X pythonw.c compile error with 10.4 or earlier deployment tar 01/08/10
       http://bugs.python.org/issue7658    created  ned.deily                     
                                                                               

Problems with attribute assignment on object instances           01/08/10
       http://bugs.python.org/issue7659    created  pakal                         
                                                                               



Issues Now Closed (40)
______________________

shutil fails when copying to NTFS in Linux                        762 days
       http://bugs.python.org/issue1545    benjamin.peterson             
       patch                                                                   

Test                                                              751 days
       http://bugs.python.org/issue1619    georg.brandl                  
                                                                               

Windows Registry issue with 2.5.2 AMD64 msi                       645 days
       http://bugs.python.org/issue2539    loewis                        
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     618 days
       http://bugs.python.org/issue2698    Daniel26                      
                                                                               

optparse print_usage(),.. methods are not documented              542 days
       http://bugs.python.org/issue3340    ezio.melotti                  
                                                                               

reference leaks in test_distutils                                 500 days
       http://bugs.python.org/issue3660    ezio.melotti                  
       patch                                                                   

_sha256 et al. encode to UTF-8 by default                          20 days
       http://bugs.python.org/issue3745    lemburg                       
       26backport                                                              

Add Option to Bind to a Local IP Address in httplib.py            464 days
       http://bugs.python.org/issue3972    gregory.p.smith               
       patch, easy                                                             

no reference for optparse methods                                 388 days
       http://bugs.python.org/issue4635    ezio.melotti                  
                                                                               

PyArg_Parse* should raise TypeError for float parsed with intege  339 days
       http://bugs.python.org/issue5080    mark.dickinson                
       patch                                                                   

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()              282 days
       http://bugs.python.org/issue5576    mark.dickinson                
       patch                                                                   

PyFrame_GetLineNumber                                             244 days
       http://bugs.python.org/issue5954    georg.brandl                  
       patch                                                                   

PyCode_NewEmpty                                                   244 days
       http://bugs.python.org/issue5959    georg.brandl                  
       patch                                                                   

Add non-command help topics to help completion of cmd.Cmd         241 days
       http://bugs.python.org/issue5991    georg.brandl                  
       patch                                                                   

make error                                                        231 days
       http://bugs.python.org/issue6067    georg.brandl                  
                                                                               

no longer possible to hash arrays                                 229 days
       http://bugs.python.org/issue6071    benjamin.peterson             
       patch                                                                   

zipfile: Invalid argument when opening zero-sized files           173 days
       http://bugs.python.org/issue6511    r.david.murray                
                                                                               

use different mechanism for pythonw on osx                        127 days
       http://bugs.python.org/issue6834    ned.deily                     
       easy                                                                    

Py3k doc: "from __future__ import division" not necessary          32 days
       http://bugs.python.org/issue7432    ezio.melotti                  
                                                                               

cPickle: stack underflow in load_pop()                             31 days
       http://bugs.python.org/issue7455    pitrou                        
       patch                                                                   

crash in str.rfind() with an invalid start value                   25 days
       http://bugs.python.org/issue7458    pitrou                        
       patch                                                                   

Implement fastsearch algorithm for rfind/rindex                    26 days
       http://bugs.python.org/issue7462    ezio.melotti                  
       patch                                                                   

GZipFile.readline too slow                                         20 days
       http://bugs.python.org/issue7471    pitrou                        
       patch                                                                   

ssl module documentation: SSLSocket.unwrap description shown twi    5 days
       http://bugs.python.org/issue7592    georg.brandl                  
                                                                               

Installation - 2 tests failed: test_cmd_line test_xmlrpc            7 days
       http://bugs.python.org/issue7601    ezio.melotti                  
                                                                               

optparse library documentation has an insignificant formatting i    2 days
       http://bugs.python.org/issue7618    ezio.melotti                  
       patch                                                                   

imaplib shouldn't use cause DeprecationWarnings in 2.6              1 days
       http://bugs.python.org/issue7619    djc                           
                                                                               

Test issue                                                          0 days
       http://bugs.python.org/issue7621    ezio.melotti                  
                                                                               

PropertyType missing in Lib/types.py                                0 days
       http://bugs.python.org/issue7623    benjamin.peterson             
                                                                               

Entity references without semicolon in HTMLParser                   2 days
       http://bugs.python.org/issue7626    r.david.murray                
                                                                               

round() doesn't work correctly in 3.1.1                             0 days
       http://bugs.python.org/issue7628    benjamin.peterson             
                                                                               

Compiling with mingw32 gcc, content of variable "extra_postargs"    0 days
       http://bugs.python.org/issue7629    r.david.murray                
                                                                               

Strange behaviour of decimal.Decimal                                0 days
       http://bugs.python.org/issue7630    mark.dickinson                
                                                                               

undefined label: bltin-file-objects                                 0 days
       http://bugs.python.org/issue7631    pitrou                        
                                                                               

next/previous links in documentation skip some sections             0 days
       http://bugs.python.org/issue7634    georg.brandl                  
                                                                               

Add a set update action to optparse                                 3 days
       http://bugs.python.org/issue7636    r.david.murray                
       patch                                                                   

Counterintuitive str.splitlines() inconsistency.                    0 days
       http://bugs.python.org/issue7638    flox                          
                                                                               

Built-in Formatter accepts undocumented presentation type           0 days
       http://bugs.python.org/issue7641    eric.smith                    
                                                                               

Fix description of the way the PythonPath Windows registry key w    0 days
       http://bugs.python.org/issue7653    r.david.murray                
       patch, needs review                                                     

PEP 374 Title usage & script redirection typo fixes                 0 days
       http://bugs.python.org/issue7655    georg.brandl                  
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 22 [patch] improve unicode methods: split() rsplit() and replace()    5 days
open    http://bugs.python.org/issue7622   

 14 Test suite emits many DeprecationWarnings when -3 is enabled      91 days
open    http://bugs.python.org/issue7092   

 13 unicode_escape codec does not escape quotes                        8 days
open    http://bugs.python.org/issue7615   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    0 days
open    http://bugs.python.org/issue7658   

  8 Merge C version of decimal into py3k.                              1 days
open    http://bugs.python.org/issue7652   

  8 "u'%c' % char" broken for chars in range '\x80'-'\xFF'             2 days
open    http://bugs.python.org/issue7649   

  8 decimal.py: type conversion in context methods                     4 days
open    http://bugs.python.org/issue7633   

  8 Patch for [ 735515 ] urllib2 should cache 301 redir              906 days
open    http://bugs.python.org/issue1755841

  7 Test issue                                                         0 days
closed  http://bugs.python.org/issue7621   

  7 Cannot use both read and readline method in same ZipExtFile obj    8 days
open    http://bugs.python.org/issue7610   




From tseaver at palladion.com  Fri Jan  8 22:09:54 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:09:54 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<hi6ibr$qag$1@ger.gmane.org>
	<ca471dc21001080759y7c708421jc8102b952052d542@mail.gmail.com>
Message-ID: <hi86v3$3j5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Jan 7, 2010 at 10:12 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> The BOM should not be seekeable if the file is opened with the proposed
>> "guess encoding from BOM" mode:  it isn't properly part of the stream at
>> all in that case.
> 
> This feels about right to me. There are still questions though:
> immediately after opening a file with a BOM, what should .tell()
> return? And regardless of that, .seek(0) should put the file in that
> same initial state.

I think the behavior should be something like:

 >>> f = open('/path/to/maybe-BOM-encoded-file', 'r', encoding='BOM')
 >>> f.tell()
 0L
 >>> f.seek(-1)
 >>> f.tell() # count of unicode chars in decoded stream
 45L
 >>> f.seek(0)
 >>> f.read(1) # read first unicode char decoded from stream.
 'A'

In other words, the BOM is not readable / seekable at all:  it is
invisible to the consumer of the decoded stream.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHnyIACgkQ+gerLs4ltQ6s3QCgznD+7FbUzfCbe5TS6OcoXjMg
rdgAoJAMEXe2xwLCIwJaZ6XA6rVyTIAi
=oXb3
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:19:10 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:19:10 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B475C72.1010207@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com>
Message-ID: <hi87gg$8ge$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

M.-A. Lemburg wrote:

> Shouldn't this encoding guessing be a separate function that you call
> on either a file or a seekable stream ?
> 
> After all, detecting encodings is just as useful to have for non-file
> streams.

Other stream sources typically have out-of-band ways to signal the
encoding:  only when reading from the filesystem do we pretty much
*have* to guess, and in that case the BOM / signature is the best
heuristic we have.  Also, some non-file streams are not seekable, and so
can't be guessed via a pre-pass.

> You'd then avoid having to stuff everything into
> a single function call and also open up the door for more complex
> application specific guess work or defaults.
> 
> The whole process would then have two steps:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_file_encoding(filename)

Filename is not enough information:  or do you mean that API to actually
open the stream?

>  2. open the file with the found encoding
> 
>   f = open(filename, encoding=encoding)
> 
> For seekable streams f, you'd have:
> 
>  1. guess encoding
> 
>   import codecs
>   encoding = codecs.guess_stream_encoding(f)
> 
>  2. wrap the stream with a reader for the found encoding
> 
>   reader_class = codecs.getreader(encoding)
>   g = reader_class(f)
> 


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoU4ACgkQ+gerLs4ltQ5o3QCeLOJ7J91E+5f66vhgu1BUhYh4
9UgAnR2IeCd0BCsPez8ZilGNHJfhRn3Y
=SoPb
-----END PGP SIGNATURE-----



From tseaver at palladion.com  Fri Jan  8 22:14:59 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:14:59 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B46F54D.9090103@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de>
Message-ID: <hi878l$7os$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:

>>> It *is* crazy, but unfortunately rather common.  Wikipedia has a good
>>> description of the issues:
>>> <http://en.wikipedia.org/wiki/UTF-8#Byte-order_mark>.  Basically, some
>>> Windows text APIs will emit a UTF-8 "BOM" in order to identify the file as
>>> being UTF-8, so it's become a convention to do that.  That's not good
>>> enough, so you need to guess the encoding as well to make sure, but if there
>>> is a BOM and you can otherwise verify that the file is probably UTF-8
>>> encoded, you should discard it.
>> That doesn't make sense. If the file isn't UTF-8 you can't see the
>> BOM, because the BOM itself is UTF-8-encoded.
> 
> I think what Glyph meant is this: if a file starts with the UTF-8
> signature, assume it's UTF-8. Then validate the assumption against the
> rest of the file also, and then process it as UTF-8. If the rest clearly
> is not UTF-8, assume that the UTF-8 signature is bogus.

If the programmer opens the file using a "guess using the BOM" encoding,
 Python should *not* attempt to verify that the file is properly
encoded:  it should check for (and consume) any BOM, and then return a
stream which uses the encoding inferred from the BOM.  Any errors should
be handled later, when characters are read, exactly as if the file had
been opened with the same encoding guessed from the BOM.

> I understood this proposal as a general processing guideline, not
> something the io library should do (but, say, a text editor).
> 
> FWIW, I'm personally in favor of using the UTF-8 signature. If people
> consider them crazy talk, that may be because UTF-8 can't possibly have
> a byte order - hence I call it a signature, not the BOM. As a signature,
> I don't consider it crazy at all. There is a long tradition of having
> magic bytes in files (executable files, Postscript, PDF, ... - see
> /etc/magic). Having a magic byte sequence for plain text to denote the
> encoding is useful and helps reducing moji-bake. This is the reason it's
> used on Windows: notepad would normally assume that text is in the ANSI
> code page, and for compatibility, it can't stop doing that. So the UTF-8
> signature gives them an exit strategy.

Agreed.  Having that marker at the start of the file makes interop with
other tools *much* easier.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHoFMACgkQ+gerLs4ltQ73dACffwUfyh6Q9vUnKYf367QFjNcU
RRMAoNuKCWEx7j+MSdTv+UjhAPynBc14
=uAX6
-----END PGP SIGNATURE-----



From eric at trueblade.com  Fri Jan  8 22:40:47 2010
From: eric at trueblade.com (Eric Smith)
Date: Fri, 8 Jan 2010 16:40:47 -0500 (EST)
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
Message-ID: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>

>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
>>
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
>
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

But what if the file were in (for example) a zip file? I think you
definitely want to have access to this functionality outside of open().

Eric.



From foom at fuhm.net  Fri Jan  8 22:49:23 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 8 Jan 2010 16:49:23 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<4B46F54D.9090103@v.loewis.de> <hi878l$7os$1@ger.gmane.org>
Message-ID: <D481E750-3EE7-4498-9959-B4E34E02FFC6@fuhm.net>

On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote:
>> I understood this proposal as a general processing guideline, not
>> something the io library should do (but, say, a text editor).
>>
>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly  
>> have
>> a byte order - hence I call it a signature, not the BOM. As a  
>> signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote  
>> the
>> encoding is useful and helps reducing moji-bake. This is the reason  
>> it's
>> used on Windows: notepad would normally assume that text is in the  
>> ANSI
>> code page, and for compatibility, it can't stop doing that. So the  
>> UTF-8
>> signature gives them an exit strategy.
>
> Agreed.  Having that marker at the start of the file makes interop  
> with
> other tools *much* easier.

Putting the BOM at the beginning of UTF-8 text files is not a good  
idea, it makes interop much *worse* on a unix system, not better.  
Without the BOM, most commands do the right thing with UTF-8 text.  
E.g. to concatenate two files:

$ cat file-1 file-2 > file-3

With a BOM at the beginning of the file, it won't work right. Of  
course, you could modify "cat" (and every other stream processing  
command) to know how to consume and emit BOMs, and omit the extra one  
that would show up in the middle of the stream...but even that can't  
work; what about:
$ (cat file-1; cat file-2) > file-3.

Should the shell now know that when you run multiple commands, it  
should eat the BOM emitted from the second command?

Basically, using a BOM in a utf-8 file is just not a good idea: it  
completely ruins interop with every standard unix tool.

This is not to say that Python shouldn't have a way to read a file  
with a UTF-8 BOM: it just shouldn't encourage you to *write* such files.

James


From mal at egenix.com  Fri Jan  8 22:51:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 08 Jan 2010 22:51:26 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi87gg$8ge$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>	<loom.20100108T153305-228@post.gmane.org>	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>	<4B475C72.1010207@egenix.com>
	<hi87gg$8ge$1@ger.gmane.org>
Message-ID: <4B47A8DE.1080401@egenix.com>

Tres Seaver wrote:
> M.-A. Lemburg wrote:
> 
>> Shouldn't this encoding guessing be a separate function that you call
>> on either a file or a seekable stream ?
> 
>> After all, detecting encodings is just as useful to have for non-file
>> streams.
> 
> Other stream sources typically have out-of-band ways to signal the
> encoding:  only when reading from the filesystem do we pretty much
> *have* to guess, and in that case the BOM / signature is the best
> heuristic we have.  Also, some non-file streams are not seekable, and so
> can't be guessed via a pre-pass.

Sure there are non-seekable file streams, but at least when
using StringIO-type streams you don't have that restriction.

An encoding detection function would provide more use in other
cases as well, so instead of hiding away the functionality in
the open() constructor, I'm suggesting to make expose it via
the codecs module.

Applications can then use it where necessary and also provide their
own defaults, using other heuristics.

>> You'd then avoid having to stuff everything into
>> a single function call and also open up the door for more complex
>> application specific guess work or defaults.
> 
>> The whole process would then have two steps:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_file_encoding(filename)
> 
> Filename is not enough information:  or do you mean that API to actually
> open the stream?

Yes. The API would open the file, guess the encoding and then
close it again. If you don't want that to happen, you could use
the second API I mentioned below on the already open file.

Note that this function could detect a lot more encodings with
reasonably high probability than just BOM-prefixed ones,
e.g. we could also add support to detect encoding declarations
such as the ones we use in Python source files.

>>  2. open the file with the found encoding
> 
>>   f = open(filename, encoding=encoding)
> 
>> For seekable streams f, you'd have:
> 
>>  1. guess encoding
> 
>>   import codecs
>>   encoding = codecs.guess_stream_encoding(f)

I forgot to mention: This API needs to position the file pointer
to the start of the first data byte.

>>  2. wrap the stream with a reader for the found encoding
> 
>>   reader_class = codecs.getreader(encoding)
>>   g = reader_class(f)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 08 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From tseaver at palladion.com  Fri Jan  8 22:59:04 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 08 Jan 2010 16:59:04 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>
	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>
	<201001081127.44044.victor.stinner@haypocalc.com>
	<loom.20100108T153305-228@post.gmane.org>
	<ca471dc21001080754kcc41a79udb1332b37240b70c@mail.gmail.com>
	<4B475C72.1010207@egenix.com> <hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <hi89r8$g7b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric Smith wrote:
>>> Shouldn't this encoding guessing be a separate function that you call
>>> on either a file or a seekable stream ?
>>>
>>> After all, detecting encodings is just as useful to have for non-file
>>> streams.
>> Other stream sources typically have out-of-band ways to signal the
>> encoding:  only when reading from the filesystem do we pretty much
>> *have* to guess, and in that case the BOM / signature is the best
>> heuristic we have.  Also, some non-file streams are not seekable, and so
>> can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

If the application expects a possibly-BOM-signature-marked file, but you
pass it mismatched garbage:

  >>> f = open('some.zip', encoding='BOM")

the error handling should be the same as if you passed any other
mismatched encoding:

  >>> f = open('some.zip', encoding='UTF8')

i.e., you discover the error when you try to read from the (non)encoded
stream, not when you open it.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktHqpwACgkQ+gerLs4ltQ7uAACeKEc+WT4TASGcVl1Hfqe6L9La
I6EAn1pJtngtLWPdothGbYB+zUabEvTW
=TjBK
-----END PGP SIGNATURE-----



From victor.stinner at haypocalc.com  Fri Jan  8 23:10:32 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 8 Jan 2010 23:10:32 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<hi87gg$8ge$1@ger.gmane.org>
	<36707.63.251.87.214.1262986847.squirrel@mail.trueblade.com>
Message-ID: <201001082310.33029.victor.stinner@haypocalc.com>

Le vendredi 08 janvier 2010 22:40:47, Eric Smith a ?crit :
> >> Shouldn't this encoding guessing be a separate function that you call
> >> on either a file or a seekable stream ?
> >>
> >> After all, detecting encodings is just as useful to have for non-file
> >> streams.
> >
> > Other stream sources typically have out-of-band ways to signal the
> > encoding:  only when reading from the filesystem do we pretty much
> > *have* to guess, and in that case the BOM / signature is the best
> > heuristic we have.  Also, some non-file streams are not seekable, and so
> > can't be guessed via a pre-pass.
> 
> But what if the file were in (for example) a zip file? I think you
> definitely want to have access to this functionality outside of open().

FYI my patch (encoding="BOM") is implemented in TextIOWrapper, and 
TextIOWrapper takes a binary stream as input, not a filename.

-- 
Victor Stinner
http://www.haypocalc.com/


From yoann.padioleau at facebook.com  Fri Jan  8 23:59:52 2010
From: yoann.padioleau at facebook.com (Yoann Padioleau)
Date: Fri, 8 Jan 2010 14:59:52 -0800
Subject: [Python-Dev] relation between Python.asdl and
 Tools/compiler/ast.txt
In-Reply-To: <4B464F1C.7020404@v.loewis.de>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>
	<4B464499.4020305@v.loewis.de>
	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>
	<4B464F1C.7020404@v.loewis.de>
Message-ID: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>


On Jan 7, 2010, at 1:16 PM, Martin v. L?wis wrote:

>>> astgen.py is not used to process asdl files; ast.txt lives right
>>> next to astgen.py. Instead, the asdl file is processed by
>>> Parser/asdl_c.py.
>> 
>> Yes, I know that. That's why I asked about the relation between
>> ast.txt and Python.adsl. If internally the parser uses the .adsl, but
>> expose as a reflection mechanism things that were generated from
>> ast.txt, then there could be a mismatch. Where does ast.txt comes
>> from ? Shouldn't it be generated itself from Python.adsl ?
> 
> What you may not be aware of is that Tools/compiler (and the
> compiler package that it builds on) are both unused and unmaintained.

I see. So if people want to analyze python code they have to use
other tools (like rope?) ? or use reflection ?

> 
> If the package stops working correctly - tough luck.
> 
>> So we would have
>> 
>> Python.adsl ----????----> ast.txt ---- astgen.py --->  ast.py
>> containing all the UnarySub, Expression, classes that represents a
>> Python AST.
> 
> No - what actually happens in Python 3.x is this: both the compiler
> package and Tools/compiler are removed.

Ok. I will then create my own ast classes generator.

Thanks.


> 
> Regards,
> Martin



From g.brandl at gmx.net  Sat Jan  9 00:10:24 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 09 Jan 2010 00:10:24 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <hi878l$7os$1@ger.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<4B46F54D.9090103@v.loewis.de>
	<hi878l$7os$1@ger.gmane.org>
Message-ID: <hi8e1n$sos$1@ger.gmane.org>

Am 08.01.2010 22:14, schrieb Tres Seaver:

>> FWIW, I'm personally in favor of using the UTF-8 signature. If people
>> consider them crazy talk, that may be because UTF-8 can't possibly have
>> a byte order - hence I call it a signature, not the BOM. As a signature,
>> I don't consider it crazy at all. There is a long tradition of having
>> magic bytes in files (executable files, Postscript, PDF, ... - see
>> /etc/magic). Having a magic byte sequence for plain text to denote the
>> encoding is useful and helps reducing moji-bake. This is the reason it's
>> used on Windows: notepad would normally assume that text is in the ANSI
>> code page, and for compatibility, it can't stop doing that. So the UTF-8
>> signature gives them an exit strategy.
> 
> Agreed.  Having that marker at the start of the file makes interop with
> other tools *much* easier.

Except if only 50% of the other tools support the signature.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From martin at v.loewis.de  Sat Jan  9 00:57:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 00:57:46 +0100
Subject: [Python-Dev] relation between Python.asdl and
	Tools/compiler/ast.txt
In-Reply-To: <F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
References: <7B83F217-4808-475E-95F2-FB64238C3ADD@facebook.com>	<4B464499.4020305@v.loewis.de>	<6A95699A-4770-4347-9BA8-2CCC853443B5@facebook.com>	<4B464F1C.7020404@v.loewis.de>
	<F1B43AEF-12E4-4C3A-80D1-DA7D33971BBF@facebook.com>
Message-ID: <4B47C67A.3020302@v.loewis.de>

> I see. So if people want to analyze python code they have to use
> other tools (like rope?) ? or use reflection ?

Correct. One such tool might be the true Python compiler, along
with the _ast module.

Regards,
Martin


From victor.stinner at haypocalc.com  Sat Jan  9 00:59:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 00:59:00 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
Message-ID: <201001090059.00848.victor.stinner@haypocalc.com>

Hi,

Thanks for all the answers! I will try to sum up all ideas here.


(1) Change default open() behaviour or make it optional?

Guido would like to add an option and keep open() unchanged. He wrote that 
checking for BOM and using system locale are too much different to be the same 
option (encoding=None).

Antoine would like to check BOM by default, because both options (system 
locale vs checking for BOM) is the same thing.

About Antoine choice (encoding=None): which file modes would check for a BOM? 
I would like to answer only the read only mode, but then open(filename, "r") 
and open(filename, "r+") would behave differently?

  => 1 point for Guido (encoding="BOM" is more explicit)

Antoine choice has the advantage of directly support UTF-8+BOM, UTF-16 and 
UTF-32 for all applications and all modules using open(filename).

  => 1 point for Antoine


(2) Check for a BOM while reading or detect it before?

Everybody agree that checking BOM is an interesting option and should not be 
limited to open().

Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
name or a binary file-like object: it returns the encoding and seek to the 
file start or just after the BOM.

I dislike this function because it requires extra file operations (open 
(optional), read() and seek()) and it doesn't work if the file is not seekable 
(eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
avoid extra file operations.

Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
any file-like object.


(3) tell() and seek() on a text file starting with a BOM

To be consistent with Antoine example:

   >>> bio = io.BytesIO(b'\xff\xfea\x00b\x00')
   >>> f = io.TextIOWrapper(bio, encoding='utf-16')
   >>> f.read()
   'ab'
   >>> f.seek(0)
   0
   >>> f.read()
   'ab'

TextIOWrapper:

 * tell() should return zero at file start,
 * seek(0) should go be to file start,
 * and the BOM should always be "ignored".

I mean:

  with open("utf8bom.txt", encoding="BOM") as fp:
     assert fp.tell() == 0   
     text = fp.read() # no BOM here
     fp.seek(0)
     assert fp.read() == text

--

About my patch:

 - BOM check is explicit: open(filebame,  encoding="BOM")
 - tell() / seek(0) works as expected
 - BOM bytes are always skipped in read() / readlines() result

Hum, I don't know if this email can be called a sum up ;-)

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 01:09:48 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 00:09:48 +0000 (UTC)
Subject: [Python-Dev] Quick sum up about open() + BOM
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <loom.20100109T010657-648@post.gmane.org>

Hello Victor,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> (1) Change default open() behaviour or make it optional?
> 
[...]
> 
> Antoine would like to check BOM by default, because both options (system 
> locale vs checking for BOM) is the same thing.

To be clear, I am not saying it is the same thing. What I think is that it would
be a mistake to use a mildly unreliable heuristic by default (the locale +
device encoding heuristic) but refuse to trust a more reliable heuristic (the
BOM-based detection algorithm).

Regards

Antoine.




From fuzzyman at voidspace.org.uk  Sat Jan  9 01:14:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 00:14:39 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <loom.20100109T010657-648@post.gmane.org>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<loom.20100109T010657-648@post.gmane.org>
Message-ID: <4B47CA6F.5000607@voidspace.org.uk>

On 09/01/2010 00:09, Antoine Pitrou wrote:
> Hello Victor,
>
> Victor Stinner<victor.stinner<at>  haypocalc.com>  writes:
>    
>> (1) Change default open() behaviour or make it optional?
>>
>>      
> [...]
>    
>> Antoine would like to check BOM by default, because both options (system
>> locale vs checking for BOM) is the same thing.
>>      
> To be clear, I am not saying it is the same thing. What I think is that it would
> be a mistake to use a mildly unreliable heuristic by default (the locale +
> device encoding heuristic) but refuse to trust a more reliable heuristic (the
> BOM-based detection algorithm).
>    

I concur. On Windows both UTF-8 and signature are very common, yet the 
platform default is the truly awful CP1252.

All the best,

Michael
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From floris.bruynooghe at gmail.com  Sat Jan  9 01:26:05 2010
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 9 Jan 2010 00:26:05 +0000
Subject: [Python-Dev] --enabled-shared broken on freebsd5?
In-Reply-To: <4B46F6D7.1050301@v.loewis.de>
References: <66d0a6e11001061314k302b4e5xb5702cd6dc46a60e@mail.gmail.com>
	<4B450CF8.8090009@v.loewis.de>
	<66d0a6e11001061608u20fa89aeyed0c707452475cc9@mail.gmail.com>
	<66d0a6e11001071911v72da8326ued1221fdfda2e2ff@mail.gmail.com>
	<4B46F6D7.1050301@v.loewis.de>
Message-ID: <20100109002605.GB13536@laurie.devork>

On Fri, Jan 08, 2010 at 10:11:51AM +0100, "Martin v. L?wis" wrote:
> Nicholas Bastin wrote:
> > I think this problem probably needs to move over to distutils-sig, as
> > it doesn't seem to be specific to the way that Python itself uses
> > distutils.
> 
> I'm fairly skeptical that anybody on distutils SIG is interested in
> details of the Python build process...

Uh, hum.  Unfounded skepticism.  ;-)
But as said filing a bug sounds better in this case.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org


From v+python at g.nevcal.com  Sat Jan  9 01:47:38 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 16:47:38 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B47D22A.8070305@g.nevcal.com>

On approximately 1/8/2010 3:59 PM, came the following characters from 
the keyboard of Victor Stinner:
> Hi,
>
> Thanks for all the answers! I will try to sum up all ideas here.

One concern I have with this implementation encoding="BOM" is that if 
there is no BOM it assumes UTF-8.  That is probably a good assumption in 
some circumstances, but not in others.

* It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
(cases where the endianness is unspecified) contain a BOM.  Hence, it 
might be that someone would expect a UTF-16LE (or any of the formats 
that don't require a BOM, rather than UTF-8), but be willing to accept 
any BOM-discriminated format.

* Potentially, this could be expanded beyond the various Unicode 
encodings... one could envision that a program whose data files 
historically were in any particular national language locale, could want 
to be enhance to accept Unicode, and could declare that they will accept 
any BOM-discriminated format, but want to default, in the absence of a 
BOM, to the original national language locale that they historically 
accepted.  That would provide a migration path for their old data files.

So the point is, that it might be nice to have 
"BOM-otherEncodingForDefault" for each other encoding that Python 
supports.  Not sure that is the right API, but I think it is expressive 
enough to handle the cases above.  Whether the cases solve actual 
problems or not, I couldn't say, but they seem like reasonable cases.

It would, of course, be nicest if OS metadata had been invented way back 
when, for all OSes, such that all text files were flagged with their 
encoding... then languages could just read the encoding and do the right 
thing! But we live in the real world, instead.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From python at mrabarnett.plus.com  Sat Jan  9 02:12:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 09 Jan 2010 01:12:28 +0000
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B47D7FC.4070703@mrabarnett.plus.com>

Glenn Linderman wrote:
> On approximately 1/8/2010 3:59 PM, came the following characters from 
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if 
> there is no BOM it assumes UTF-8.  That is probably a good assumption in 
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32 
> (cases where the endianness is unspecified) contain a BOM.  Hence, it 
> might be that someone would expect a UTF-16LE (or any of the formats 
> that don't require a BOM, rather than UTF-8), but be willing to accept 
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode 
> encodings... one could envision that a program whose data files 
> historically were in any particular national language locale, could want 
> to be enhance to accept Unicode, and could declare that they will accept 
> any BOM-discriminated format, but want to default, in the absence of a 
> BOM, to the original national language locale that they historically 
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have 
> "BOM-otherEncodingForDefault" for each other encoding that Python 
> supports.  Not sure that is the right API, but I think it is expressive 
> enough to handle the cases above.  Whether the cases solve actual 
> problems or not, I couldn't say, but they seem like reasonable cases.
> 
> It would, of course, be nicest if OS metadata had been invented way back 
> when, for all OSes, such that all text files were flagged with their 
> encoding... then languages could just read the encoding and do the right 
> thing! But we live in the real world, instead.
> 
What about listing the possible encodings? It would try each in turn
until it found one where the BOM matched or had no BOM:

     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')

or is that taking it too far?


From martin at v.loewis.de  Sat Jan  9 02:23:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 09 Jan 2010 02:23:07 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47CA6F.5000607@voidspace.org.uk>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<loom.20100109T010657-648@post.gmane.org>
	<4B47CA6F.5000607@voidspace.org.uk>
Message-ID: <4B47DA7B.6050505@v.loewis.de>

>>> Antoine would like to check BOM by default, because both options
>>> (system locale vs checking for BOM) is the same thing.
>>> 
>> To be clear, I am not saying it is the same thing. What I think is 
>> that it would be a mistake to use a mildly unreliable heuristic by
>> default (the locale + device encoding heuristic) but refuse to
>> trust a more reliable heuristic (the BOM-based detection
>> algorithm).
>> 
> 
> I concur. On Windows both UTF-8 and signature are very common, yet
> the platform default is the truly awful CP1252.

While I would support combining BOM detection in the case where a file
is opened for reading and no encoding is specified, I see two problems:
a) if a seek operations is performed before having looked at the BOM,
   no determination would have been made
b) what encoding should it use on writing?

Regards,
Martin



From v+python at g.nevcal.com  Sat Jan  9 02:49:12 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 08 Jan 2010 17:49:12 -0800
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <4B47E098.6030703@g.nevcal.com>

On approximately 1/8/2010 5:12 PM, came the following characters from 
the keyboard of MRAB:
> Glenn Linderman wrote:
>> On approximately 1/8/2010 3:59 PM, came the following characters from 
>> the keyboard of Victor Stinner:
>>> Hi,
>>>
>>> Thanks for all the answers! I will try to sum up all ideas here.
>>
>> One concern I have with this implementation encoding="BOM" is that if 
>> there is no BOM it assumes UTF-8.  That is probably a good assumption 
>> in some circumstances, but not in others.
>>
>> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE 
>> encoded files include a BOM.  It is only required that UTF-16 and 
>> UTF-32 (cases where the endianness is unspecified) contain a BOM.  
>> Hence, it might be that someone would expect a UTF-16LE (or any of 
>> the formats that don't require a BOM, rather than UTF-8), but be 
>> willing to accept any BOM-discriminated format.
>>
>> * Potentially, this could be expanded beyond the various Unicode 
>> encodings... one could envision that a program whose data files 
>> historically were in any particular national language locale, could 
>> want to be enhance to accept Unicode, and could declare that they 
>> will accept any BOM-discriminated format, but want to default, in the 
>> absence of a BOM, to the original national language locale that they 
>> historically accepted.  That would provide a migration path for their 
>> old data files.
>>
>> So the point is, that it might be nice to have 
>> "BOM-otherEncodingForDefault" for each other encoding that Python 
>> supports.  Not sure that is the right API, but I think it is 
>> expressive enough to handle the cases above.  Whether the cases solve 
>> actual problems or not, I couldn't say, but they seem like reasonable 
>> cases.
>>
>> It would, of course, be nicest if OS metadata had been invented way 
>> back when, for all OSes, such that all text files were flagged with 
>> their encoding... then languages could just read the encoding and do 
>> the right thing! But we live in the real world, instead.
>>
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
>
>     my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

That sounds very flexible -- but in net effect it would only make 
illegal a subset of the BOM-containing encodings (those not listed) 
without making legal any additional encodings other than the non-BOM 
encoding.  Whether prohibiting a subset of BOM-containing encodings is a 
useful use case, I couldn't say... but my goal would be to included as 
many different file encodings on input as possible: without a BOM, that 
is exactly 1 (unless there are other heuristics), with a BOM, it is 
1+all-BOM-containing encodings.  Your scheme would permit numbers of 
encodings accepted to vary between 1 and 1+all-BOM-containing encodings.

(I think everyone can agree there are 5 different byte sequences that 
can be called a Unicode BOM.  The likelihood of them appearing in any 
other text encoding created by mankind depends on those other encodings 
-- but it is not impossible.  It is truly up to the application to 
decide whether BOM detection could potentially conflict with files in 
some other encoding that would be acceptable to the application.)

So I think it is taking it further than I can see value in, but I'm 
willing to be convinced otherwise.  I see only a need for detecting BOM, 
and specifying a default encoding to be used if there is no BOM.  Note 
that it might be nice to have a specification for using current 
encoding=None heuristic -- perhaps encoding="BOM-None" per my originally 
proposed syntax.  But I'm still not saying that is the best syntax.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking



From ncoghlan at gmail.com  Sat Jan  9 04:07:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 09 Jan 2010 13:07:12 +1000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B476196.4080409@mrabarnett.plus.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<BC1A2A2E-1DAE-4D2C-821D-77A1D902622A@twistedmatrix.com>	<ca471dc21001072021s73db375cx6cbd0f39b7849745@mail.gmail.com>	<201001081127.44044.victor.stinner@haypocalc.com>
	<4B476196.4080409@mrabarnett.plus.com>
Message-ID: <4B47F2E0.7090403@gmail.com>

MRAB wrote:
> Maybe there should also be a way of determining what encoding it decided
> it was, so that you can then write a new file in that same encoding.

I thought of that question as well - the f.encoding attribute on the
opened file should be sufficient.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From regebro at gmail.com  Sat Jan  9 06:48:36 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 06:48:36 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47E098.6030703@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com> <4B47D7FC.4070703@mrabarnett.plus.com>
	<4B47E098.6030703@g.nevcal.com>
Message-ID: <319e029f1001082148h5cee2c0bn76f70a17766ca69e@mail.gmail.com>

It seems to me that when opening a file, the following is the only
flow that makes sense for the typical opening of a file flow:

if encoding is not None:
   use encoding
elif file has BOM:
   use BOM
else:
   use system default

And hence a encoding='BOM' isn't needed there. Although I'm trying to
come up with usecases that doesn't work with this, I can't. :)

BUT

When writing things are not so easy though. Apparently some encodings
require a BOM to be written, but others do not, but allow it, and some
has no byte order mark. So there you have to be able to write the BOM,
or not. And that's either a new parameter, because you can't use
encoding='BOM' since you need to specify the encoding as well, or a
new method.

I would suggest a BOM parameter, and maybe a method as  well.

BOM=None|True|False

Where "None" means a sane default behaviour, that is write a BOM if
the encoding require it.
"True" means write a BOM if the encoding *supports* it.
"False" means Don't write a BOM even if the encoding requires it
(because I know what I'm doing)

if 'w' in mode: # But not 'r' or 'a'
    if BOM == True and encoding in (ENCODINGS THAT ALLOW BOM):
        write_bom = True
    elif BOM == False:
       write_bom = False
    elif BOM == None and encoding in (ENCODINGS THAT REQUIRE BOM):
          write_bom = True
    else:
          write_bom = False
else:
    write_bom = False

For reading this parameter could either be a noop, or possibly change
the behavior somehow, if a usecase where that makes sense can be
imagined.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From walter at livinglogic.de  Sat Jan  9 11:51:57 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 11:51:57 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <4B485FCD.2040901@livinglogic.de>

On 09.01.10 01:47, Glenn Linderman wrote:

> On approximately 1/8/2010 3:59 PM, came the following characters from
> the keyboard of Victor Stinner:
>> Hi,
>>
>> Thanks for all the answers! I will try to sum up all ideas here.
> 
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.  That is probably a good assumption in
> some circumstances, but not in others.
> 
> * It is not required that UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE
> encoded files include a BOM.  It is only required that UTF-16 and UTF-32
> (cases where the endianness is unspecified) contain a BOM.  Hence, it
> might be that someone would expect a UTF-16LE (or any of the formats
> that don't require a BOM, rather than UTF-8), but be willing to accept
> any BOM-discriminated format.
> 
> * Potentially, this could be expanded beyond the various Unicode
> encodings... one could envision that a program whose data files
> historically were in any particular national language locale, could want
> to be enhance to accept Unicode, and could declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted.  That would provide a migration path for their old data files.
> 
> So the point is, that it might be nice to have
> "BOM-otherEncodingForDefault" for each other encoding that Python
> supports.  Not sure that is the right API, but I think it is expressive
> enough to handle the cases above.  Whether the cases solve actual
> problems or not, I couldn't say, but they seem like reasonable cases.

This is doable with the currect API. Simply define a codec search
function that handles all encoding names that start with "BOM-" and pass
the "otherEncodingForDefault" part along to the codec.

> It would, of course, be nicest if OS metadata had been invented way back
> when, for all OSes, such that all text files were flagged with their
> encoding... then languages could just read the encoding and do the right
> thing! But we live in the real world, instead.

Servus,
   Walter


From walter at livinglogic.de  Sat Jan  9 12:18:33 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Sat, 09 Jan 2010 12:18:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001081140.28124.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
Message-ID: <4B486609.2050804@livinglogic.de>

Victor Stinner wrote:
> Le vendredi 08 janvier 2010 10:10:23, Martin v. L?wis a ?crit :
>>> Builtin open() function is unable to open an UTF-16/32 file starting with
>>> a BOM if the encoding is not specified (raise an unicode error). For an
>>> UTF-8 file starting with a BOM, read()/readline() returns also the BOM
>>> whereas the BOM should be "ignored".
>> It depends. If you use the utf-8-sig encoding, it *will* ignore the
>> UTF-8 signature.
> 
> Sure, but it means that you only use UTF-8+BOM files. If you get UTF-8 and 
> UTF-8+BOM files, you have to to detect the encoding (not an easy job) or to 
> remove the BOM after the first read (much harder if you use a module like 
> ConfigParser or csv).
> 
>>> Since my proposition changes the result TextIOWrapper.read()/readline()
>>> for files starting with a BOM, we might introduce an option to open() to
>>> enable the new behaviour. But is it really needed to keep the backward
>>> compatibility?
>> Absolutely. And there is no need to produce a new option, but instead
>> use the existing options: define an encoding that auto-detects the
>> encoding from the family of BOMs. Maybe you call it encoding="sniff".
> 
> Good idea, I choosed open(filename, encoding="BOM").

On the surface this looks like there's an encoding named "BOM", but 
looking at your patch I found that the check is still done in 
TextIOWrapper. IMHO the best approach would to the implement a *real* 
codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
the IO library. It could even be developed as a standalone project and 
published in the Cheeseshop.

To see how something like this can be done, take a look at the UTF-16 
codec, that switches to bigendian or littleendian mode depending on the 
first read/decode call.

Servus,
    Walter







From victor.stinner at haypocalc.com  Sat Jan  9 13:37:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:37:06 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47DA7B.6050505@v.loewis.de>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47CA6F.5000607@voidspace.org.uk> <4B47DA7B.6050505@v.loewis.de>
Message-ID: <201001091337.06596.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:23:07, Martin v. L?wis a ?crit :
> While I would support combining BOM detection in the case where a file
> is opened for reading and no encoding is specified, I see two problems:
> a) if a seek operations is performed before having looked at the BOM,
>    no determination would have been made

TextIOWrapper doesn't support seek to an arbitrary byte. It uses "cookie" 
which is an opaque value. Reuse a cookie from another file or an old cookie is 
forbidden (but it doesn't raise an error). This is not specific to the BOM 
checking: the problem already exist for encodings using a BOM (eg. UTF-16).

> b) what encoding should it use on writing?

Don't change anything to writing.

With Antoince choice: open('file.txt', 'w', encoding=None) continue to use the 
actual heuristic (os.device_encoding() or system locale).

With Guido choice, encoding="BOM": it raises an error, because BOM check is 
not supported when writing into a file. How could the BOM be checked when 
creating a new (empty) file!?

-- 
Victor Stinner
http://www.haypocalc.com/


From mal at egenix.com  Sat Jan  9 13:45:58 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 09 Jan 2010 13:45:58 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <201001090059.00848.victor.stinner@haypocalc.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
Message-ID: <4B487A86.4020603@egenix.com>

Victor Stinner wrote:
> (2) Check for a BOM while reading or detect it before?
> 
> Everybody agree that checking BOM is an interesting option and should not be 
> limited to open().
> 
> Marc-Andre proposed a codecs.guess_file_encoding() function accepting a file 
> name or a binary file-like object: it returns the encoding and seek to the 
> file start or just after the BOM.
> 
> I dislike this function because it requires extra file operations (open 
> (optional), read() and seek()) and it doesn't work if the file is not seekable 
> (eg. a pipe). I prefer to check for a BOM at first read in TextIOWrapper to 
> avoid extra file operations.
> 
> Note: I implemented the BOM check in TextIOWrapper; so it's already usable for 
> any file-like object.

Yes, but the implementation is limited to just BOM checking
and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

With a codecs module function we could easily extend the
encoding detection to more file types, e.g. XML files,
Python source code files, etc. that use other mechanisms
for defining the encoding.

BTW: I haven't looked at your implementation, but what happens
when your BOM check fails ? Will the implementation add the
already read bytes back to a buffer ?

This rollback action is the only reason for needing a
seekable stream in codecs.guess_stream_encoding().

Another point to consider:

AFAIK, we currently have a moratorium on changes to Python
builtins. How does that match up with the proposed changes ?

Using a new codec like Walter suggested would move the
implementation into the stdlib for which doesn't the
moratorium doesn't apply.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From victor.stinner at haypocalc.com  Sat Jan  9 13:54:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 13:54:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D22A.8070305@g.nevcal.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
Message-ID: <201001091354.17239.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 01:47:38, vous avez ?crit :
> One concern I have with this implementation encoding="BOM" is that if
> there is no BOM it assumes UTF-8.

If no BOM is found, it fallback to the current heuristic: os.device_encoding() 
or system local.

> (...) Hence, it might be that someone would expect a UTF-16LE (or any of 
> the formats that don't require a BOM, rather than UTF-8), but be willing 
> to accept any BOM-discriminated format.
> (...) declare that they will accept
> any BOM-discriminated format, but want to default, in the absence of a
> BOM, to the original national language locale that they historically
> accepted

You mean "if there is a BOM, use it, otherwise fallback to a specific 
charset"? How could it be declared? Maybe:

   open("file.txt", check_bom=True, encoding="UTF16-LE")
   open("file.txt", check_bom=True, encoding="latin1")

About falling back to UTF-8, it would be written:

   open("file.txt", check_bom=True, encoding="UTF-8")

As explained before, check_bom=True is only accepted for read only file mode.

Well, why not. This is a third choice for my point (1) :-) It's between Guido 
and Antoine choice, and I like it because we can fallback to UTF-8 instead of 
the dummy system locale: Windows users will be happy to be able to use UTF-8 
:-) I prefer to fallback to a fixed encoding then depending on the system 
locale.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:34:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:34:17 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B47D7FC.4070703@mrabarnett.plus.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B47D22A.8070305@g.nevcal.com>
	<4B47D7FC.4070703@mrabarnett.plus.com>
Message-ID: <201001091434.17521.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 02:12:28, MRAB a ?crit :
> What about listing the possible encodings? It would try each in turn
> until it found one where the BOM matched or had no BOM:
> 
>      my_file = open(filename, 'r', encoding='UTF-8-sig|UTF-16|UTF-8')
>
> or is that taking it too far?

Yes, you're taking it foo far :-) Checking BOM is reliable, whereas *guessing* 
the charset only using the byte stream can only be an heuristic. Guess a 
charset is a complex problem, they are 3rd party library to do that, like the 
chardet project.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:38:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:38:43 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B486609.2050804@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <201001091438.43576.victor.stinner@haypocalc.com>

Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
> > Good idea, I choosed open(filename, encoding="BOM").
> 
> On the surface this looks like there's an encoding named "BOM", but
> looking at your patch I found that the check is still done in
> TextIOWrapper. IMHO the best approach would to the implement a *real*
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
> the IO library. It could even be developed as a standalone project and
> published in the Cheeseshop.

Why not, this is another solution to the point (2) (Check for a BOM while 
reading or detect it before?). Which encoding would be used if there is not 
BOM? UTF-8 sounds like a good choice.

-- 
Victor Stinner
http://www.haypocalc.com/


From victor.stinner at haypocalc.com  Sat Jan  9 14:50:28 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 9 Jan 2010 14:50:28 +0100
Subject: [Python-Dev] Quick sum up about open() + BOM
In-Reply-To: <4B487A86.4020603@egenix.com>
References: <201001090059.00848.victor.stinner@haypocalc.com>
	<4B487A86.4020603@egenix.com>
Message-ID: <201001091450.28497.victor.stinner@haypocalc.com>

Hi,

Le samedi 09 janvier 2010 13:45:58, vous avez ?crit :
> > Note: I implemented the BOM check in TextIOWrapper; so it's already
> > usable for any file-like object.
> 
> Yes, but the implementation is limited to just BOM checking
> and thus only supports UTF-8-SIG, UTF-16 and UTF-32.

Sure, but that's already better than no BOM check :-) It looks like many 
people would apprecite UTF-8-SIG detection, since this encoding is common on 
Windows.

> BTW: I haven't looked at your implementation, but what happens
> when your BOM check fails ? Will the implementation add the
> already read bytes back to a buffer ?

My implementation is done between buffer.read() and decoder.decode(data). If 
there is a BOM: set the encoding and remove the BOM bytes from the byte 
string. Otherwise, use another algorithm to choose the encoding and leave the 
byte string unchanged.

It can be seen as a codec: it works like UTF-16 and UTF-32 codecs ;-)

> AFAIK, we currently have a moratorium on changes to Python
> builtins. How does that match up with the proposed changes ?

Oh yes, I forgot the moratorium. In all solutions, some of them don't change 
the API. Eg. Antoine proposed to leave the API unchanged: open(file) => 
open(file) :-) I don't know if it's compatible with the moratorium or not.

-- 
Victor Stinner
http://www.haypocalc.com/


From solipsis at pitrou.net  Sat Jan  9 16:05:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 15:05:45 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
Message-ID: <loom.20100109T160248-501@post.gmane.org>

Walter D?rwald <walter <at> livinglogic.de> writes:
> 
> On the surface this looks like there's an encoding named "BOM", but 
> looking at your patch I found that the check is still done in 
> TextIOWrapper. IMHO the best approach would to the implement a *real* 
> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
> the IO library. It could even be developed as a standalone project and 
> published in the Cheeseshop.

Sorry but this is missing the point. The point here is to improve the open()
function. I'm sure people who know about encodings are able to install the
chardet library or even whip up their own BOM detection routine...


Antoine.




From benjamin at python.org  Sat Jan  9 18:29:33 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 11:29:33 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>

On behalf of the Python development team, I'm gleeful to announce the second
alpha release of Python 2.7.

Python 2.7 is scheduled to be the last major version in the 2.x series.  It
includes many features that were first released in Python 3.1.  The faster io
module, the new nested with statement syntax, improved float repr, and the
memoryview object have been backported from 3.1. Other features include an
ordered dictionary implementation, unittests improvements, and support for ttk
Tile in Tkinter.  For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7 visit:

     http://www.python.org/download/releases/2.7/

Please note that this is a development release, intended as a preview of new
features for the community, and is thus not suitable for production use.

The 2.7 documentation can be found at:

     http://docs.python.org/2.7

Please consider trying Python 2.7 with your code and reporting any bugs you may
notice to:

     http://bugs.python.org


Have fun!

--
Benjamin Peterson
2.7 Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7's contributors)


From kmtracey at gmail.com  Sat Jan  9 19:48:12 2010
From: kmtracey at gmail.com (Karen Tracey)
Date: Sat, 9 Jan 2010 13:48:12 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>

On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:

> On behalf of the Python development team, I'm gleeful to announce the
> second
> alpha release of Python 2.7.
>
>
Well yay.  Django's test suite (1242 tests) runs with just one failure on
the 2.7 alpha 2 level, and that looks to be likely due to the improved
string/float rounding so not really a problem, just a difference.  That's
down from 104 failures and 40 errors with 2.7 alpha 1.

Note on the website page http://www.python.org/download/releases/2.7/ the
"Change log for this release" link is still pointing to the alpha 1
changelog.

Thanks,
Karen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100109/d5c06f60/attachment-0007.htm>

From benjamin at python.org  Sat Jan  9 19:51:11 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 12:51:11 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <1afaf6161001091051kb1ba08q9b96094af16c12fa@mail.gmail.com>

2010/1/9 Karen Tracey <kmtracey at gmail.com>:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>>
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>
> Well yay.? Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.? That's
> down from 104 failures and 40 errors with 2.7 alpha 1.

Excellent!

>
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Thanks. I'll fix that.

>



-- 
Regards,
Benjamin


From skip at pobox.com  Sat Jan  9 21:00:44 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:00:44 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
Message-ID: <19272.57452.972234.643008@montanaro.dyndns.org>

How much of the Unladen Swallow cPickle speedups have been incorporated into
2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
two versions I currently care about at work - we will skip 2.5 entirely).
It appears some of their speedups may have already been merged to trunk, but
I'm not sure how much.  If a patch to merge this to 2.7 is already under
consideration I won't look at it, but I interpreted Collin Winter's response
to my query on the u-s mailing list that not everything has been done yet.

Thx,

Skip



From martin at v.loewis.de  Sat Jan  9 21:09:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 09 Jan 2010 21:09:27 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T160248-501@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
Message-ID: <4B48E277.9010408@v.loewis.de>

Antoine Pitrou wrote:
> Walter D?rwald <walter <at> livinglogic.de> writes:
>> On the surface this looks like there's an encoding named "BOM", but 
>> looking at your patch I found that the check is still done in 
>> TextIOWrapper. IMHO the best approach would to the implement a *real* 
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to 
>> the IO library. It could even be developed as a standalone project and 
>> published in the Cheeseshop.
> 
> Sorry but this is missing the point. The point here is to improve the open()
> function. I'm sure people who know about encodings are able to install the
> chardet library or even whip up their own BOM detection routine...

How does the requirement that it be implemented as a codec miss the
point?

FWIW, I agree with Walter that if it is provided through the encoding=
argument, it should be a codec. If it is built into the open function
(for whatever reason), it must be provided by some other parameter.

I do see the point that it becomes available to end users only when
released as part of Python. However, this *also* means that applications
won't be using it for another three years or so, since they'll have to
support older Python versions as well (unless it is integrated in the
case where no encoding is specified).

Regards,
Martin


From pjenvey at underboss.org  Sat Jan  9 21:09:29 2010
From: pjenvey at underboss.org (Philip Jenvey)
Date: Sat, 9 Jan 2010 12:09:29 -0800
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <19272.57452.972234.643008@montanaro.dyndns.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>


On Jan 9, 2010, at 12:00 PM, skip at pobox.com wrote:

> How much of the Unladen Swallow cPickle speedups have been incorporated into
> 2.7 & 3.1?  I'm working on trying to develop patches for 2.4 and 2.6 (the
> two versions I currently care about at work - we will skip 2.5 entirely).
> It appears some of their speedups may have already been merged to trunk, but
> I'm not sure how much.  If a patch to merge this to 2.7 is already under
> consideration I won't look at it, but I interpreted Collin Winter's response
> to my query on the u-s mailing list that not everything has been done yet.

They've documented their upstream patches here:

http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

--
Philip Jenvey



From skip at pobox.com  Sat Jan  9 21:21:20 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:21:20 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 & 3.1
In-Reply-To: <7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<7855D115-33BE-47F6-BDA9-ABC0A4D7E759@underboss.org>
Message-ID: <19272.58688.173011.412712@montanaro.dyndns.org>


    Philip> They've documented their upstream patches here:

    Philip> http://code.google.com/p/unladen-swallow/wiki/UpstreamPatches

Thanks.  That will help immensely.

Skip



From solipsis at pitrou.net  Sat Jan  9 21:28:12 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:28:12 +0000 (UTC)
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
Message-ID: <loom.20100109T212555-508@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Sorry but this is missing the point. The point here is to improve the open()
> > function. I'm sure people who know about encodings are able to install the
> > chardet library or even whip up their own BOM detection routine...
> 
> How does the requirement that it be implemented as a codec miss the
> point?

If we want it to be the default, it must be able to fallback on the current
locale-based algorithm if no BOM is found. I don't think it would be easy for a
codec to do that.

> FWIW, I agree with Walter that if it is provided through the encoding=
> argument, it should be a codec. If it is built into the open function
> (for whatever reason), it must be provided by some other parameter.

Why not simply encoding=None? The default value should provide the most useful
behaviour possible. Forcing users to choose between two different autodetection
strategies (encoding=None and another one) is a little insane IMO.

Regards

Antoine.




From solipsis at pitrou.net  Sat Jan  9 21:32:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 9 Jan 2010 20:32:27 +0000 (UTC)
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
References: <19272.57452.972234.643008@montanaro.dyndns.org>
Message-ID: <loom.20100109T213033-976@post.gmane.org>

<skip <at> pobox.com> writes:
> 
> If a patch to merge this to 2.7 is already under
> consideration I won't look at it,

Why won't you look at it? :)
Actually, if these patches are to be merged someone should certainly look at
them, and do the (possibly) remaining work.

http://bugs.python.org/issue5683
http://bugs.python.org/issue5671

Thank you

Antoine.




From skip at pobox.com  Sat Jan  9 21:43:42 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 9 Jan 2010 14:43:42 -0600
Subject: [Python-Dev] Unladen cPickle speedups in 2.7 &amp; 3.1
In-Reply-To: <loom.20100109T213033-976@post.gmane.org>
References: <19272.57452.972234.643008@montanaro.dyndns.org>
	<loom.20100109T213033-976@post.gmane.org>
Message-ID: <19272.60030.25319.269597@montanaro.dyndns.org>

>>>>> "Antoine" == Antoine Pitrou <solipsis at pitrou.net> writes:

    Antoine> <skip <at> pobox.com> writes:
    >> 
    >> If a patch to merge this to 2.7 is already under
    >> consideration I won't look at it,

    Antoine> Why won't you look at it? :)

I meant I wouldn't look at developing one.

Skip


From regebro at gmail.com  Sat Jan  9 23:14:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 9 Jan 2010 23:14:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>

On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Right. It seems like encoding=None is the right way to go there.
encoding='BOM' would probably only work if 'BOM' isn't an encoding but
a special tag, which is ugly.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From fuzzyman at voidspace.org.uk  Sun Jan 10 00:25:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 09 Jan 2010 23:25:18 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<319e029f1001091414w7cb5e296w5c5e38e1a23ab3d5@mail.gmail.com>
Message-ID: <4B49105E.303@voidspace.org.uk>

On 09/01/2010 22:14, Lennart Regebro wrote:
> On Sat, Jan 9, 2010 at 21:28, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>    
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
>>      
> Right. It seems like encoding=None is the right way to go there.
> encoding='BOM' would probably only work if 'BOM' isn't an encoding but
> a special tag, which is ugly.
>
>    
I would rather see it as the default behavior for open without an 
encoding specified.

I know Guido has expressed a preference against this so I won't continue 
to flog it.

The current behavior however is that we have a 'guessing' algorithm 
based on the platform default. Currently if you open a text file in read 
mode that has a UTF-8 signature, but the platform default is something 
other than UTF-8, then we open the file using what is likely to be the 
incorrect encoding. Looking for the signature seems to be better 
behaviour in that case.

All the best,

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Sun Jan 10 00:40:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 10 Jan 2010 00:40:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <loom.20100109T212555-508@post.gmane.org>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
Message-ID: <4B4913DF.7050801@v.loewis.de>

>> How does the requirement that it be implemented as a codec miss the
>> point?
> 
> If we want it to be the default, it must be able to fallback on the current
> locale-based algorithm if no BOM is found. I don't think it would be easy for a
> codec to do that.

Yes - however, Victor currently apparently *doesn't* want it to be the
default, but wants the user to specify encoding="BOM". If so, it isn't
the default, and it is easy to implement as a codec.

>> FWIW, I agree with Walter that if it is provided through the encoding=
>> argument, it should be a codec. If it is built into the open function
>> (for whatever reason), it must be provided by some other parameter.
> 
> Why not simply encoding=None?

I don't mind. Please re-read Walter's message - it only said that
*if* this is activated through encoding="BOM", *then* it must be
a codec, and could be on PyPI. I don't think Walter was talking about
the case "it is not activated through encoding='BOM'" *at all*.

> The default value should provide the most useful
> behaviour possible. Forcing users to choose between two different autodetection
> strategies (encoding=None and another one) is a little insane IMO.

That wouldn't disturb me much. There are a lot of things in that area
that are a little insane, starting with Microsoft Windows :-)

Regards,
Martin


From henning.vonbargen at arcor.de  Sun Jan 10 12:10:02 2010
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 10 Jan 2010 12:10:02 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <mailman.2987.1263079529.28904.python-dev@python.org>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
Message-ID: <4B49B58A.4040103@arcor.de>

If Python should support BOM when reading text files,
it should also be able to *write* such files.

An encoding="BOM" argument wouldn't help here, because
it does not specify which encoding to use actually:
UFT-8, UTF-16-LE or what?

That would be a point against encoding="BOM" and
pro an additional keyword argument "use_bom" or whatever
with the following values:

None: default (old) behaviour: don't handle BOM at all

True: reading: expect BOM (raising an exception if it's
                missing). The encoding argument must be None
                or it must match the encoding implied by the
                BOM
       writing: write a BOM. The encoding argument must be
                one of the UTF encodings.
False: reading: If a BOM is present, use it to determine the
                file encoding. The encoding argument must
                be None or it must match the encoding implied by
                the BOM. (*)
                Otherwise, use the encoding argument to determine
                the encoding.
        writing: do not write a BOM. Use the encoding argument.

(*) This is a question of taste. I think some people would prefer
     a fourth value "AUTO" instead, or to swap the behaviour of
     None and False.

Henning

P.S. To make things worse, I have sometimes seen XML files with a
UTF-8 BOM, but an XML encoding declaration of "iso-8859-1".
For such files, whatever you guess will be wrong anyway...


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Sun Jan 10 12:21:48 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 10 Jan 2010 12:21:48 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B49B58A.4040103@arcor.de>
References: <mailman.2987.1263079529.28904.python-dev@python.org>
	<4B49B58A.4040103@arcor.de>
Message-ID: <319e029f1001100321pfed5deatd7483a49be343ba7@mail.gmail.com>

On Sun, Jan 10, 2010 at 12:10, Henning von Bargen
<henning.vonbargen at arcor.de> wrote:
> If Python should support BOM when reading text files,
> it should also be able to *write* such files.

That's what I thought too. Turns out the UTF-16 does write such a
mark. You also have the constants in the codecs module, so you can
write the utf-16-le BOM and then use the utf-16-le encoding if you
want to be sure you write utf-16-le, and the same with BE, of course.

I still think now using BOM's when determining the file format can be
seen as a bug, though, so I don't think the API needs to change at
all.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From brett at python.org  Sun Jan 10 19:51:26 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 10:51:26 -0800
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
Message-ID: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>

Nick Coghlan thought I should forward this to python-dev so people are aware
of this change and why it occurred (plus it indirectly informs Guido I
finally finished the work).

---------- Forwarded message ----------
From: brett.cannon <python-checkins at python.org>
Date: Sat, Jan 9, 2010 at 18:56
Subject: [Python-checkins] r77402 - in python/trunk:
Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS Python/_warnings.c
To: python-checkins at python.org


Author: brett.cannon
Date: Sun Jan 10 03:56:19 2010
New Revision: 77402

Log:
DeprecationWarning is now silent by default.

This was originally suggested by Guido, discussed on the stdlib-sig mailing
list, and given the OK by Guido directly to me. What this change essentially
means is that Python has taken a policy of silencing warnings that are only
of interest to developers by default. This should prevent users from seeing
warnings which are triggered by an application being run against a new
interpreter before the app developer has a chance to update their code.

Closes issue #7319. Thanks to Antoine Pitrou, Ezio Melotti, and Brian Curtin
for helping with the issue.


Modified:
  python/trunk/Doc/library/warnings.rst
  python/trunk/Lib/test/test_ascii_formatd.py
  python/trunk/Lib/test/test_exceptions.py
  python/trunk/Lib/warnings.py
  python/trunk/Misc/NEWS
  python/trunk/Python/_warnings.c

Modified: python/trunk/Doc/library/warnings.rst
==============================================================================
--- python/trunk/Doc/library/warnings.rst       (original)
+++ python/trunk/Doc/library/warnings.rst       Sun Jan 10 03:56:19 2010
@@ -59,7 +59,7 @@
 | :exc:`UserWarning`               | The default category for :func:`warn`.
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`DeprecationWarning`        | Base category for warnings about
deprecated   |
-|                                  | features.
        |
+|                                  | features (ignored by default).
       |
 +----------------------------------+-----------------------------------------------+
 | :exc:`SyntaxWarning`             | Base category for warnings about
dubious      |
 |                                  | syntactic features.
        |
@@ -89,6 +89,9 @@
 standard warning categories.  A warning category must always be a subclass
of
 the :exc:`Warning` class.

+.. versionchanged:: 2.7
+   :exc:`DeprecationWarning` is ignored by default.
+

 .. _warning-filter:

@@ -148,14 +151,6 @@
 :mod:`warnings` module parses these when it is first imported (invalid
options
 are ignored, after printing a message to ``sys.stderr``).

-The warnings that are ignored by default may be enabled by passing
:option:`-Wd`
-to the interpreter. This enables default handling for all warnings,
including
-those that are normally ignored by default. This is particular useful for
-enabling ImportWarning when debugging problems importing a developed
package.
-ImportWarning can also be enabled explicitly in Python code using::
-
-   warnings.simplefilter('default', ImportWarning)
-

 .. _warning-suppress:

@@ -226,6 +221,37 @@
 entries from the warnings list before each new operation).


+Updating Code For New Versions of Python
+----------------------------------------
+
+Warnings that are only of interest to the developer are ignored by default.
As
+such you should make sure to test your code with typically ignored warnings
+made visible. You can do this from the command-line by passing
:option:`-Wd`
+to the interpreter (this is shorthand for :option:`-W default`).  This
enables
+default handling for all warnings, including those that are ignored by
default.
+To change what action is taken for encountered warnings you simply change
what
+argument is passed to :option:`-W`, e.g. :option:`-W error`. See the
+:option:`-W` flag for more details on what is possible.
+
+To programmatically do the same as :option:`-Wd`, use::
+
+  warnings.simplefilter('default')
+
+Make sure to execute this code as soon as possible. This prevents the
+registering of what warnings have been raised from unexpectedly influencing
how
+future warnings are treated.
+
+Having certain warnings ignored by default is done to prevent a user from
+seeing warnings that are only of interest to the developer. As you do not
+necessarily have control over what interpreter a user uses to run their
code,
+it is possible that a new version of Python will be released between your
+release cycles.  The new interpreter release could trigger new warnings in
your
+code that were not there in an older interpreter, e.g.
+:exc:`DeprecationWarning` for a module that you are using. While you as a
+developer want to be notified that your code is using a deprecated module,
to a
+user this information is essentially noise and provides no benefit to them.
+
+
 .. _warning-functions:

 Available Functions

Modified: python/trunk/Lib/test/test_ascii_formatd.py
==============================================================================
--- python/trunk/Lib/test/test_ascii_formatd.py (original)
+++ python/trunk/Lib/test/test_ascii_formatd.py Sun Jan 10 03:56:19 2010
@@ -4,6 +4,7 @@

 import unittest
 from test_support import check_warnings, run_unittest, cpython_only
+import warnings


 class FormatDeprecationTests(unittest.TestCase):
@@ -17,6 +18,7 @@
        buf = create_string_buffer(' ' * 100)

        with check_warnings() as w:
+            warnings.simplefilter('default')
            PyOS_ascii_formatd(byref(buf), sizeof(buf), '%+.10f',
                               c_double(10.0))
            self.assertEqual(buf.value, '+10.0000000000')

Modified: python/trunk/Lib/test/test_exceptions.py
==============================================================================
--- python/trunk/Lib/test/test_exceptions.py    (original)
+++ python/trunk/Lib/test/test_exceptions.py    Sun Jan 10 03:56:19 2010
@@ -309,6 +309,7 @@
        # BaseException.__init__ triggers a deprecation warning.
        exc = BaseException("foo")
        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('default')
            self.assertEquals(exc.message, "foo")
        self.assertEquals(len(w), 1)
        self.assertEquals(w[0].category, DeprecationWarning)

Modified: python/trunk/Lib/warnings.py
==============================================================================
--- python/trunk/Lib/warnings.py        (original)
+++ python/trunk/Lib/warnings.py        Sun Jan 10 03:56:19 2010
@@ -383,8 +383,8 @@
 # Module initialization
 _processoptions(sys.warnoptions)
 if not _warnings_defaults:
-    simplefilter("ignore", category=PendingDeprecationWarning, append=1)
-    simplefilter("ignore", category=ImportWarning, append=1)
+    for cls in (DeprecationWarning, PendingDeprecationWarning,
ImportWarning):
+        simplefilter("ignore", category=cls, append=True)
    bytes_warning = sys.flags.bytes_warning
    if bytes_warning > 1:
        bytes_action = "error"

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sun Jan 10 03:56:19 2010
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------

+- Issue #7319: Silence DeprecationWarning by default.
+
 - Issue #2335: Backport set literals syntax from Python 3.x.

 Library

Modified: python/trunk/Python/_warnings.c
==============================================================================
--- python/trunk/Python/_warnings.c     (original)
+++ python/trunk/Python/_warnings.c     Sun Jan 10 03:56:19 2010
@@ -85,10 +85,10 @@

    default_action = get_warnings_attr("defaultaction");
    if (default_action == NULL) {
-       if (PyErr_Occurred()) {
-           return NULL;
-       }
-       return _default_action;
+        if (PyErr_Occurred()) {
+            return NULL;
+        }
+        return _default_action;
    }

    Py_DECREF(_default_action);
@@ -202,12 +202,12 @@

    mod_str = PyString_AsString(filename);
    if (mod_str == NULL)
-           return NULL;
+        return NULL;
    len = PyString_Size(filename);
    if (len < 0)
        return NULL;
    if (len >= 3 &&
-       strncmp(mod_str + (len - 3), ".py", 3) == 0) {
+            strncmp(mod_str + (len - 3), ".py", 3) == 0) {
        module = PyString_FromStringAndSize(mod_str, len-3);
    }
    else {
@@ -251,7 +251,7 @@

    name = PyObject_GetAttrString(category, "__name__");
    if (name == NULL)  /* XXX Can an object lack a '__name__' attribute? */
-           return;
+        return;

    f_stderr = PySys_GetObject("stderr");
    if (f_stderr == NULL) {
@@ -341,7 +341,7 @@
        rc = already_warned(registry, key, 0);
        if (rc == -1)
            goto cleanup;
-       else if (rc == 1)
+        else if (rc == 1)
            goto return_none;
        /* Else this warning hasn't been generated before. */
    }
@@ -492,9 +492,9 @@
    /* Setup filename. */
    *filename = PyDict_GetItemString(globals, "__file__");
    if (*filename != NULL) {
-           Py_ssize_t len = PyString_Size(*filename);
+            Py_ssize_t len = PyString_Size(*filename);
        const char *file_str = PyString_AsString(*filename);
-           if (file_str == NULL || (len < 0 && PyErr_Occurred()))
+            if (file_str == NULL || (len < 0 && PyErr_Occurred()))
            goto handle_error;

        /* if filename.lower().endswith((".pyc", ".pyo")): */
@@ -506,10 +506,10 @@
                tolower(file_str[len-1]) == 'o'))
        {
            *filename = PyString_FromStringAndSize(file_str, len-1);
-               if (*filename == NULL)
-                       goto handle_error;
-           }
-           else
+            if (*filename == NULL)
+                goto handle_error;
+        }
+        else
            Py_INCREF(*filename);
    }
    else {
@@ -536,8 +536,8 @@
            else {
                /* embedded interpreters don't have sys.argv, see bug
#839151 */
                *filename = PyString_FromString("__main__");
-                   if (*filename == NULL)
-                       goto handle_error;
+                if (*filename == NULL)
+                    goto handle_error;
            }
        }
        if (*filename == NULL) {
@@ -839,26 +839,29 @@
 static PyObject *
 init_filters(void)
 {
-    PyObject *filters = PyList_New(3);
+    PyObject *filters = PyList_New(4);
    const char *bytes_action;
    if (filters == NULL)
        return NULL;

    PyList_SET_ITEM(filters, 0,
+                    create_filter(PyExc_DeprecationWarning, "ignore"));
+    PyList_SET_ITEM(filters, 1,
                    create_filter(PyExc_PendingDeprecationWarning,
"ignore"));
-    PyList_SET_ITEM(filters, 1, create_filter(PyExc_ImportWarning,
"ignore"));
+    PyList_SET_ITEM(filters, 2, create_filter(PyExc_ImportWarning,
"ignore"));
    if (Py_BytesWarningFlag > 1)
        bytes_action = "error";
    else if (Py_BytesWarningFlag)
        bytes_action = "default";
    else
        bytes_action = "ignore";
-    PyList_SET_ITEM(filters, 2, create_filter(PyExc_BytesWarning,
+    PyList_SET_ITEM(filters, 3, create_filter(PyExc_BytesWarning,
                    bytes_action));

    if (PyList_GET_ITEM(filters, 0) == NULL ||
        PyList_GET_ITEM(filters, 1) == NULL ||
-        PyList_GET_ITEM(filters, 2) == NULL) {
+        PyList_GET_ITEM(filters, 2) == NULL ||
+        PyList_GET_ITEM(filters, 3) == NULL) {
        Py_DECREF(filters);
        return NULL;
     }
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db6da5fd/attachment-0007.htm>

From victor.stinner at haypocalc.com  Sun Jan 10 20:22:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 10 Jan 2010 20:22:10 +0100
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
Message-ID: <201001102022.10259.victor.stinner@haypocalc.com>

Hi,

Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
> Nick Coghlan thought I should forward this to python-dev so people are
>  aware of this change and why it occurred (plus it indirectly informs Guido
>  I finally finished the work).

Thanks to copy this mail to the python-dev mailing list.

What is the recommanded way to get the previous behaviour (print deprecation 
warnings once)? Is there a command line option? Is it "-W default"?

-- 
Victor Stinner
http://www.haypocalc.com/


From benjamin at python.org  Sun Jan 10 20:23:54 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 13:23:54 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r77402 - in python/trunk:
	Doc/library/warnings.rst Lib/test/test_ascii_formatd.py
	Lib/test/test_exceptions.py Lib/warnings.py Misc/NEWS
	Python/_warnings.c
In-Reply-To: <201001102022.10259.victor.stinner@haypocalc.com>
References: <4b4941df.0967f10a.71e8.6c41SMTPIN_ADDED@mx.google.com>
	<bbaeab101001101051w26831d2bp7f83989fbbc2487b@mail.gmail.com>
	<201001102022.10259.victor.stinner@haypocalc.com>
Message-ID: <1afaf6161001101123g366d80dbjeaa80f1a632605e2@mail.gmail.com>

2010/1/10 Victor Stinner <victor.stinner at haypocalc.com>:
> Hi,
>
> Le dimanche 10 janvier 2010 19:51:26, Brett Cannon a ?crit :
>> Nick Coghlan thought I should forward this to python-dev so people are
>> ?aware of this change and why it occurred (plus it indirectly informs Guido
>> ?I finally finished the work).
>
> Thanks to copy this mail to the python-dev mailing list.
>
> What is the recommanded way to get the previous behaviour (print deprecation
> warnings once)? Is there a command line option? Is it "-W default"?

That commit conveniently adds documentation answering that question. :)



-- 
Regards,
Benjamin


From nas at arctrix.com  Sun Jan 10 20:30:09 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:30:09 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <hid9s1$i05$1@ger.gmane.org>

Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm gleeful to announce
> the second alpha release of Python 2.7.

Thanks to everyone who contributed.

> Python 2.7 is scheduled to be the last major version in the 2.x
> series.

Has this really been decided already? Maybe I missed it.  In my
opinion, it does Python's reputation harm to make such a statement.
Conservative users (probably the vast majority of Python users)
don't like to hear that software they are considering using is
nearing the end of its life.  What does it gain us to announce that
the 2.x branch is dead aside from bugfixes?

I propose that the 2.x branch be treated like 2.x.y branches: as
long as there is sufficient volunteer labour, it should continue to
live.  In order to avoid wasted development effort, it would be
prudent to announce that unless a 2.8 release manager steps up,
whatever is committed to the 2.x branch after the 2.7 release may
never get released.

Said another way, it's okay for the Python developers to decide to
abandon 2.x and put their efforts into 3.x. It's not okay for them
to prevent others from continuing to work on 2.x or to somehow make
2.x look worse so 3.x looks better. Python 3 needs to stand on its
own terms and I'm confident it can.

Best regards,

  Neil



From brett at python.org  Sun Jan 10 21:09:08 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:09:08 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>

On Sun, Jan 10, 2010 at 11:30, Neil Schemenauer <nas at arctrix.com> wrote:

> Benjamin Peterson <benjamin at python.org> wrote:
> > On behalf of the Python development team, I'm gleeful to announce
> > the second alpha release of Python 2.7.
>
> Thanks to everyone who contributed.
>
> > Python 2.7 is scheduled to be the last major version in the 2.x
> > series.
>
> Has this really been decided already? Maybe I missed it.


More or less. It was first discussed at the language summit last year and
has come up here a couple of times. If needed we can make it official in
terms of lifetime of 2.7, etc. at the language summit this year.


>  In my
> opinion, it does Python's reputation harm to make such a statement.
> Conservative users (probably the vast majority of Python users)
> don't like to hear that software they are considering using is
> nearing the end of its life.  What does it gain us to announce that
> the 2.x branch is dead aside from bugfixes?
>
> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.
>
> Said another way, it's okay for the Python developers to decide to
> abandon 2.x and put their efforts into 3.x. It's not okay for them
> to prevent others from continuing to work on 2.x or to somehow make
> 2.x look worse so 3.x looks better. Python 3 needs to stand on its
> own terms and I'm confident it can.
>

I don't think ending the 2.x series at 2.7 makes it look bad compared to
3.2; it's simply the end of a development line like any other software
project. I suspect 2.7 will have a protracted bugfix window because so much
code runs on 2.x exclusively at the moment. And if core developers want to
continue to backport fixes past two years  from release they can (or however
long we decide to officially support 2.7).

No one is saying people still can't work on the code, just that python-dev
as an entity is not going to focus its effort on the 2.x series anymore and
people should not rely upon us to continue to focus new development efforts
in that series. If there are core developers who want to continue to do
bugfix releases then that's fine and I am happy to flag patches as needing
backports and let other's do the work after the standard two year
maintenance cycle, but I know I do not want to be held accountable as a core
developer to keep the 2.x going indefinitely. Maintaining four branches is
enough of a reason in my book to not keep the 2.x series going.

If there really is an outcry on this we can re-visit the issue, but as of
right now we need to move forward at some point and 2.7 seems like that good
point.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/4bff0434/attachment-0007.htm>

From ncoghlan at gmail.com  Sun Jan 10 21:23:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 11 Jan 2010 06:23:27 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A373F.9050601@gmail.com>

Neil Schemenauer wrote:
> In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

The announcement is precisely to avoid the situation where people commit
new features to the 2.x main line of development (after the 2.7
maintenance branch is created) in the expectation that they will be
released as part of a hypothetical 2.8 release.

Whether that info needs to be in each and every 2.7 announcement...
probably not. It isn't really info for users of Python, just for
developers of Python.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Sun Jan 10 21:25:29 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 12:25:29 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
Message-ID: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>

Michael has given me the hg transition/stdlib time slot at the language
summit this year. In regards to that I plan to lead a discussion on:

* where we are at w/ the Hg transition (Dirkjan should be there and I did a
blog post on this topic recently:
http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
* argparse (PEP 389)
* brief mention on still wanting to break out the stdlib from CPython
* an official policy on extension modules? (i.e. must have a pure Python
implementation which can mean a ctypes implementation unless given an
explicit waiver)

That's everything from a stdlib perspective. I have half-baked ideas, but
nothing concrete enough to bring up unless people really want to discuss
stuff like how to potentially standardize module deprecation warnings, etc.

In terms of me personally, I do plan to bring up at some point during the
summit these points which don't squarely fit during my time slot:

* an official unofficial policy on how new proposed features should come to
us (i.e. working code to python-ideas with a shell of a PEP that includes
abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
* any changes needed to the issue tracker to help with the workflow? (stage
field seems like a failed experiment and we now have several effective
triage people who can help w/ guiding changes)

If there is something missing from the stdlib discussion that you think
should be brought up at the summit let me know. And if there is something
here you want to discuss before the summit let me know and I can start a
separate thread on it.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/f16584e2/attachment-0007.htm>

From dirkjan at ochtman.nl  Sun Jan 10 22:52:00 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 10 Jan 2010 22:52:00 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:25, Brett Cannon <brett at python.org> wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)

Unfortunately my flight doesn't land until about the time the
Mercurial slot ends, so while I'll be there later on that afternoon
for sprinting or questions or anything, I won't be at the actual
Mercurial talk at the summit.

Cheers,

Dirkjan


From fuzzyman at voidspace.org.uk  Sun Jan 10 23:27:58 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 10 Jan 2010 22:27:58 +0000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<ea2499da1001101352t1111adbcs505fb2adc034a37f@mail.gmail.com>
Message-ID: <4B4A546E.8010808@voidspace.org.uk>

On 10/01/2010 21:52, Dirkjan Ochtman wrote:
> On Sun, Jan 10, 2010 at 21:25, Brett Cannon<brett at python.org>  wrote:
>    
>> Michael has given me the hg transition/stdlib time slot at the language
>> summit this year. In regards to that I plan to lead a discussion on:
>> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
>> blog post on this topic recently:
>> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
>>      
> Unfortunately my flight doesn't land until about the time the
> Mercurial slot ends, so while I'll be there later on that afternoon
> for sprinting or questions or anything, I won't be at the actual
> Mercurial talk at the summit.
>
>    
We will probably be in 'open discussion' when you arrive so it will 
still be good to hear from you on the topic and there maybe questions 
that can't be answered until you arrive... :-)

Michael

> Cheers,
>
> Dirkjan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog




From martin at v.loewis.de  Mon Jan 11 00:02:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:02:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <hid9s1$i05$1@ger.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
Message-ID: <4B4A5C69.7090007@v.loewis.de>

> I propose that the 2.x branch be treated like 2.x.y branches: as
> long as there is sufficient volunteer labour, it should continue to
> live.  In order to avoid wasted development effort, it would be
> prudent to announce that unless a 2.8 release manager steps up,
> whatever is committed to the 2.x branch after the 2.7 release may
> never get released.

I think it's more difficult than that. It also takes developer effort
to *commit* to the trunk. So if the trunk continued to live, would
it still be ok if I closed a 2.x feature request as "won't fix", or
only committed the new feature to the 3.x branch?

As for old branches: they *don't* live in the way you claim (i.e.
remain open with changes potentially just not released). Instead,
at some point, they are frozen to bug-fix only, then to security-fix
only, and then to no change at all.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 00:07:58 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 00:07:58 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A373F.9050601@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>
Message-ID: <4B4A5DCE.3070909@v.loewis.de>

> The announcement is precisely to avoid the situation where people commit
> new features to the 2.x main line of development (after the 2.7
> maintenance branch is created) in the expectation that they will be
> released as part of a hypothetical 2.8 release.
> 
> Whether that info needs to be in each and every 2.7 announcement...
> probably not. It isn't really info for users of Python, just for
> developers of Python.

I think the announcement is indeed also for the users, and I would
prefer it to stay in the release announcements, so that people will
know for sure (and speak up) before the 2.7 release happens.

As for decisions: I don't think there was an official BDFL pronouncent,
but I recall Guido posting a message close to that, proposing that 2.7
will be a release that will see bug fix releases for an indefinite
period of time (where indefinite != infinite). This was shortly after
him proposing that perhaps we shouldn't make a 2.7 release at all, and
stop at 2.6.

As for such a decision giving a bad light on Python: I don't think that
will be the case. Instead, I hear many users surprised for how long
we have been maintaining to parallel versions - "that must have taken
a lot of effort". So everybody will likely understand that enough is
enough.

Regards,
Martin


From benjamin at python.org  Mon Jan 11 01:07:35 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 18:07:35 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>

Consider this program:

class Descr(object):
    def __init__(self, name):
        self.name = name
    def __set__(self, instance, what):
        instance.__dict__[self.name] = what

class X(object):
    attr = Descr("attr")

x = X()
print(x.attr)
x.attr = 42
print(x.attr)

It gives in output:

<__main__.Descr object at 0x7fe1c9b28150>
42

The documentation [1] says that Descr is a data descriptor because it
defines the __set__ method. It also states that data descriptors
always override the value in the instance dictionary. So, the second
line should also be the descriptor object according to the
documentation.

My question is: Is this a doc bug or a implementation bug? If the
former, it will be the description of a data descriptor much less
consistent, since it will require that a __get__ method be present,
too. If the latter, the fix may break some programs relying on the
ability to "cache" a value in the instance dictionary.

[1] http://docs.python.org/reference/datamodel#invoking-descriptors


-- 
Regards,
Benjamin


From amauryfa at gmail.com  Mon Jan 11 01:51:09 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 11 Jan 2010 01:51:09 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>

Hi,

2010/1/11 Benjamin Peterson <benjamin at python.org>:
> Consider this program:
>
> class Descr(object):
> ? ?def __init__(self, name):
> ? ? ? ?self.name = name
> ? ?def __set__(self, instance, what):
> ? ? ? ?instance.__dict__[self.name] = what
>
> class X(object):
> ? ?attr = Descr("attr")
>
> x = X()
> print(x.attr)
> x.attr = 42
> print(x.attr)
>
> It gives in output:
>
> <__main__.Descr object at 0x7fe1c9b28150>
> 42
>
> The documentation [1] says that Descr is a data descriptor because it
> defines the __set__ method. It also states that data descriptors
> always override the value in the instance dictionary. So, the second
> line should also be the descriptor object according to the
> documentation.
>
> My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
>
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

Quoting the documentation:
"""Normally, data descriptors define both __get__() and __set__(),
while non-data descriptors have just the __get__() method.
"""
Your example is neither a data descriptor nor a non-data descriptor...

The thing that worries me a bit is the "x.attr" returning the Descr
object. Descriptors should remain at the class level, and instance
should only see values. I'd prefer an AttributeError in this case.

-- 
Amaury Forgeot d'Arc


From benjamin at python.org  Mon Jan 11 02:00:32 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 10 Jan 2010 19:00:32 -0600
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<e27efe131001101651y68e1da25je2a8d02f5c62ef19@mail.gmail.com>
Message-ID: <1afaf6161001101700u21006708l2ef62bfa257e4ce7@mail.gmail.com>

2010/1/10 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...

See the footnote: http://docs.python.org/reference/datamodel#id7

>
> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.

Far too late for that, I'm afraid.



-- 
Regards,
Benjamin


From nas at arctrix.com  Mon Jan 11 02:44:57 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 19:44:57 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
Message-ID: <20100111014457.GA5289@arctrix.com>

On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> I don't think ending the 2.x series at 2.7 makes it look bad
> compared to 3.2; it's simply the end of a development line like
> any other software project. I suspect 2.7 will have a protracted
> bugfix window because so much code runs on 2.x exclusively at the
> moment.

I would guess over 99% of all Python code written doesn't run on
Python 3.  Given that, I think it is premature to close the door on
new major versions of Python 2.x. Also, we as a project should be
careful not to present the image that Python 2.x will not be
supported in the future.

> If there really is an outcry on this we can re-visit the issue,
> but as of right now we need to move forward at some point and 2.7
> seems like that good point.

I think that's bad PR.  If I had a successful product, I would not
announce its end of life just to see how many customers scream and
then decide if I should devote more resources to continue
maintaining it.

IMHO, the release notes should say something like:

     After the Python 2.7 release, the focus of Python development
     will be on Python 3.  There will continue to be maintainance
     releases of Python 2.x.


trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil


From tjreedy at udel.edu  Mon Jan 11 03:26:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Jan 2010 21:26:34 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <hie28p$joo$1@ger.gmane.org>

On 1/10/2010 8:44 PM, Neil Schemenauer wrote:
> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
>> I don't think ending the 2.x series at 2.7 makes it look bad
>> compared to 3.2; it's simply the end of a development line like
>> any other software project. I suspect 2.7 will have a protracted
>> bugfix window because so much code runs on 2.x exclusively at the
>> moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.

If the removal of old features had been done in the 2.x series, as once 
planned (Guido originally proposed removing the old meaning of int / int 
in 2.5) the same more or less would be true of 2.7. It is past time for 
other old and now duplicated features to be removed also.

   Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>
>> If there really is an outcry on this we can re-visit the issue,
>> but as of right now we need to move forward at some point and 2.7
>> seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.

Python is not being ended, but upgraded (with bloating duplications 
removed). Think of 3.1 as 2.8 with a new name.

tjr



From lrekucki at gmail.com  Mon Jan 11 04:26:48 2010
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Mon, 11 Jan 2010 04:26:48 +0100
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
Message-ID: <dfbefb091001101926l366043f8mb95f196ba14f9c9@mail.gmail.com>

> Date: Mon, 11 Jan 2010 01:51:09 +0100
> From: "Amaury Forgeot d'Arc" <amauryfa at gmail.com>
> To: Benjamin Peterson <benjamin at python.org>
> Cc: Python Dev <python-dev at python.org>
> Subject: Re: [Python-Dev] Data descriptor doc/implementation
> ? ? ? ?inconsistency
> Message-ID:
> ? ? ? ?<e27efe131001101651y68e1da25je2a8d02f5c62ef19 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> 2010/1/11 Benjamin Peterson <benjamin at python.org>:
>> Consider this program:
>>
>> class Descr(object):
>> ? ?def __init__(self, name):
>> ? ? ? ?self.name = name
>> ? ?def __set__(self, instance, what):
>> ? ? ? ?instance.__dict__[self.name] = what
>>
>> class X(object):
>> ? ?attr = Descr("attr")
>>
>> x = X()
>> print(x.attr)
>> x.attr = 42
>> print(x.attr)
>>
>> It gives in output:
>>
>> <__main__.Descr object at 0x7fe1c9b28150>
>> 42
>>
>> The documentation [1] says that Descr is a data descriptor because it
>> defines the __set__ method. It also states that data descriptors
>> always override the value in the instance dictionary. So, the second
>> line should also be the descriptor object according to the
>> documentation.
>>
>> My question is: Is this a doc bug or a implementation bug? If the
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>
> Quoting the documentation:
> """Normally, data descriptors define both __get__() and __set__(),
> while non-data descriptors have just the __get__() method.
> """
> Your example is neither a data descriptor nor a non-data descriptor...
Actually, there is this footnote [1]:

""" A descriptor can define any combination of __get__(), __set__()
and __delete__(). If it does not define __get__(), then accessing the
attribute even on an instance will return the descriptor object
itself. If the descriptor defines __set__() and/or __delete__(), it is
a data descriptor; if it defines neither, it is a non-data descriptor.
"""

Which would mean Descr is actually a data descriptor without a
__get__(), so x.attr should always return the descriptor object itself
(at least in the docs).

> The thing that worries me a bit is the "x.attr" returning the Descr
> object. Descriptors should remain at the class level, and instance
> should only see values. I'd prefer an AttributeError in this case.
>
> --
> Amaury Forgeot d'Arc

[1] http://docs.python.org/reference/datamodel#id7
-- 
Lukasz Rekucki


From brett at python.org  Mon Jan 11 04:46:04 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 19:46:04 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
Message-ID: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>

On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 12:09:08PM -0800, Brett Cannon wrote:
> > I don't think ending the 2.x series at 2.7 makes it look bad
> > compared to 3.2; it's simply the end of a development line like
> > any other software project. I suspect 2.7 will have a protracted
> > bugfix window because so much code runs on 2.x exclusively at the
> > moment.
>
> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x.


Well yeah; Python 3.0 is just over a year old with 3.1 -- the first robust
3.x release - is about six months old. From the beginning uptake was
expected to take years, not months, so saying that 3.x is not popular enough
seems premature.


> Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.
>

No one has said bugfixes will cease.


>
> > If there really is an outcry on this we can re-visit the issue,
> > but as of right now we need to move forward at some point and 2.7
> > seems like that good point.
>
> I think that's bad PR.  If I had a successful product, I would not
> announce its end of life just to see how many customers scream and
> then decide if I should devote more resources to continue
> maintaining it.
>
>
I never said that I wanted to make this announcement specifically to provoke
outcries. I said if there happened to be outcries we could possibly visit
the issue again. Basically I was being considerate and trying to leave the
door open to discuss things in the future even though I don't see the
situation changing.


> IMHO, the release notes should say something like:
>
>     After the Python 2.7 release, the focus of Python development
>     will be on Python 3.  There will continue to be maintainance
>     releases of Python 2.x.
>

No because that suggests new features will be coming to 2.x which is not
going to happen. If you want to say there will be continual bugfix releases
for 2.7 as is par the course for Python and that the number of bugfix
releases might be more than normal then I am okay with that.


>
>
> trying-to-head-off-the-python-is-dying-meme-ly y'rs Neil
>

That came and went already a couple months ago when we discussed stopping at
2.6 instead of 2.7 (see the various threads at
http://mail.python.org/pipermail/python-dev/2009-November/thread.html).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/db586677/attachment-0007.htm>

From nas at arctrix.com  Mon Jan 11 05:05:07 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:05:07 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
Message-ID: <20100111040507.GA7200@arctrix.com>

On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> >     After the Python 2.7 release, the focus of Python development
> >     will be on Python 3.  There will continue to be maintainance
> >     releases of Python 2.x.
> 
> No because that suggests new features will be coming to 2.x which is not
> going to happen. If you want to say there will be continual bugfix releases
> for 2.7 as is par the course for Python and that the number of bugfix
> releases might be more than normal then I am okay with that.

Are you are saying that if someone steps up to merge the Unladen
Swallow features into a 2.8 release and someone also steps up to cut
the release that they will be prevented from doing so?

Also, are you presuming to channel the BDFL or was this dicussed
somewhere other than python-dev? Maybe I'm overreacting but I get
the feeling that the larger and less active segment of the Python
develpment team hasn't been involved in these decisions.

Best regards,

  Neil


From nas at arctrix.com  Mon Jan 11 05:17:54 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 22:17:54 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5C69.7090007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
Message-ID: <20100111041754.GB7200@arctrix.com>

On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
> [...] would it still be ok if I closed a 2.x feature request as
> "won't fix", or only committed the new feature to the 3.x branch?

Yes.  Non-bugfix development on 2.x would optional (i.e. done by
people who want to spend the time). Since language changes are now
out (an initiative I completely support), the majority of non-bugfix
changes would be optimizations and platform porting.

Regards,

  Neil


From brett at python.org  Mon Jan 11 06:06:15 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 21:06:15 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111040507.GA7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
Message-ID: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>

On Sun, Jan 10, 2010 at 20:05, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 07:46:04PM -0800, Brett Cannon wrote:
> > On Sun, Jan 10, 2010 at 17:44, Neil Schemenauer <nas at arctrix.com> wrote:
> > >     After the Python 2.7 release, the focus of Python development
> > >     will be on Python 3.  There will continue to be maintainance
> > >     releases of Python 2.x.
> >
> > No because that suggests new features will be coming to 2.x which is not
> > going to happen. If you want to say there will be continual bugfix
> releases
> > for 2.7 as is par the course for Python and that the number of bugfix
> > releases might be more than normal then I am okay with that.
>
> Are you are saying that if someone steps up to merge the Unladen
> Swallow features into a 2.8 release and someone also steps up to cut
> the release that they will be prevented from doing so?
>
>
I honestly don't know, but it's a possibility just like any other new
feature request. If people start taking the carrots we have added to 3.x and
backporting them to keep the 2.x series alive you are essentially making the
3.x DOA by negating its benefits which I personally don't agree with.


> Also, are you presuming to channel the BDFL or was this dicussed
> somewhere other than python-dev?


This was discussed; see the November 2009 threads. Guido actually suggested
ending the 2.x branch at 2.6 until people spoke up about the amount of stuff
already backported to 2.7 from 3.x because it was being assumed to be the
end of the series to warrant keeping it to help transition to 2.7.


> Maybe I'm overreacting but I get
> the feeling that the larger and less active segment of the Python
> develpment team hasn't been involved in these decisions.
>

This all came up in November from the 3rd through the 6th (four days) over a
ton of email traffic. This was not a snap decision but a heated discussion
where even Guido participated that culminated with the decision to stop at
2.7 for the 2.x series; this was not a smoked-filled room decision. I'm
sorry if people missed it when they weren't looking, but python-dev is
supposed to be the place to watch for this kind of stuff. I don't know how
else to bring this stuff to people's attention short of also on
python-committers, but developers are basically expected to be on both lists
so I don't see where anyone did anything wrong in terms of informing
developers.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/18c3bcd7/attachment-0007.htm>

From nas at arctrix.com  Mon Jan 11 06:27:13 2010
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 10 Jan 2010 23:27:13 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
Message-ID: <20100111052713.GA8211@arctrix.com>

On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> If people start taking the carrots we have added to 3.x and
> backporting them to keep the 2.x series alive you are essentially
> making the 3.x DOA by negating its benefits which I personally
> don't agree with.

I think we have got to the heart of our disagreement. Assume that
some superhuman takes all the backwards compatible goodies from 3.x
and merges them into 2.x. If that really would kill off Python 3
then I would proclaim it a project that deserved to die. Why should
the backwards incompatible changes be endured if they cannot justify
their existance by the benefit they provide?

I guess I have more confidence in Python 3 than you do. I don't see
why Python 2.x needs to be artificially limited so that Python 3 can
benefit.

Regards,

  Neil


From brett at python.org  Mon Jan 11 07:30:49 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 10 Jan 2010 22:30:49 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
Message-ID: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>

On Sun, Jan 10, 2010 at 21:27, Neil Schemenauer <nas at arctrix.com> wrote:

> On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:
> > If people start taking the carrots we have added to 3.x and
> > backporting them to keep the 2.x series alive you are essentially
> > making the 3.x DOA by negating its benefits which I personally
> > don't agree with.
>
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x. If that really would kill off Python 3
> then I would proclaim it a project that deserved to die. Why should
> the backwards incompatible changes be endured if they cannot justify
> their existance by the benefit they provide?
>
>
When I said "carrots" I meant everything that makes Python 3 the best
version of Python, including the backward-incompatible changes.

I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.
>

I don't view it as artificial but simply a focusing of the development team
on what has been determined as the future of Python by Guido and letting go
of the past.

IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev
saying "Python 3 is the future, but we are keeping the old Python 2.x around
because we don't have *that* much faith in the future we have laid out".
That's poison to Python 3 by showing a lack of confidence in the direction
that the BDFL and python-dev as a group has chosen. Now I could be wrong and
there could actually be a large number of active contributors who want to
keep the 2.x series going, but based on the discussion that occurred the
last time this came up I believe the guys who are keeping Python running are
ready to move on.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100110/36448e5f/attachment-0007.htm>

From regebro at gmail.com  Mon Jan 11 08:08:14 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 08:08:14 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <319e029f1001102308p5de87cber2131f3aec1abc9f0@mail.gmail.com>

On Mon, Jan 11, 2010 at 06:27, Neil Schemenauer <nas at arctrix.com> wrote:
> I think we have got to the heart of our disagreement. Assume that
> some superhuman takes all the backwards compatible goodies from 3.x
> and merges them into 2.x.

The superhumans of core developers pretty much already have. That's
pretty much 2.7 you are talking about. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From chrism at plope.com  Mon Jan 11 08:27:03 2010
From: chrism at plope.com (Chris McDonough)
Date: Mon, 11 Jan 2010 02:27:03 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com>
Message-ID: <4B4AD2C7.1050703@plope.com>

Brett Cannon wrote:
> IMO keeping Python 2.x around past 2.7 is the equivalent of python-dev 
> saying "Python 3 is the future, but we are keeping the old Python 2.x 
> around because we don't have *that* much faith in the future we have 
> laid out". That's poison to Python 3 by showing a lack of confidence in 
> the direction that the BDFL and python-dev as a group has chosen. Now I 
> could be wrong and there could actually be a large number of active 
> contributors who want to keep the 2.x series going, but based on the 
> discussion that occurred the last time this came up I believe the guys 
> who are keeping Python running are ready to move on.

I think this is mostly just a branding issue.  Once the folks who have 
historically kept Python 2 running really *do* move on, there will be other 
folks who will step up and become maintainers out of necessity: those stuck on 
old platforms, permanent stragglers, people who for whatever reason actually 
like Python 2 better, etc.  It's just going to happen, there's really nothing 
anybody can do about it.

I don't think there's anything wrong with this.  If such a group of folks wants 
to get together and create another Python 2.X release, there's nothing stopping 
them from doing so except the above branding declaration.  I'd personally not 
close the door on allowing these folks to call such an effort "Python 2".

- C



From martin at v.loewis.de  Mon Jan 11 09:06:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:06:16 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A5C69.7090007@v.loewis.de>
	<20100111041754.GB7200@arctrix.com>
Message-ID: <4B4ADBF8.3030803@v.loewis.de>

Neil Schemenauer wrote:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time). 

I think *that* would give a very bad impression of Python. Depending
whom you deal with, the new feature you want may or may not get
added to the code base. Contributors would feel even more stranded
than they do now, where it may take several years to get a patch
reviewed, as you then could submit a patch, and pray that a comitter
reviews it who believes in future 2.x releases.

The point of setting policies is that it gives every user (contributors,
committers, and "end-user" developers) a reliable foundation for
expectations.

Regards,
Martin


From arcriley at gmail.com  Mon Jan 11 09:06:10 2010
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 11 Jan 2010 03:06:10 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AD2C7.1050703@plope.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com> 
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com> 
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com>
	<bbaeab101001102230l784dc5d2t665ffed0148f2c94@mail.gmail.com> 
	<4B4AD2C7.1050703@plope.com>
Message-ID: <bad82a81001110006n3cacd953g98fb25e96330ee89@mail.gmail.com>

after all these years, some people still run AmigaOS.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/4fdcbb4e/attachment-0007.htm>

From martin at v.loewis.de  Mon Jan 11 09:11:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:11:25 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111014457.GA5289@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
Message-ID: <4B4ADD2D.6070906@v.loewis.de>

> I would guess over 99% of all Python code written doesn't run on
> Python 3.  Given that, I think it is premature to close the door on
> new major versions of Python 2.x. Also, we as a project should be
> careful not to present the image that Python 2.x will not be
> supported in the future.

Why that? It is a fact: 2.x will not be supported, in some future.
Should we lie to users?

>      After the Python 2.7 release, the focus of Python development
>      will be on Python 3.  There will continue to be maintainance
>      releases of Python 2.x.

It shouldn't say that, because it wouldn't be true.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 09:18:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 09:18:30 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <4B4ADED6.5080207@v.loewis.de>

> I guess I have more confidence in Python 3 than you do. I don't see
> why Python 2.x needs to be artificially limited so that Python 3 can
> benefit.

Because it takes too much time. Too much of my time, but apparently
also too much of other people's time.

Of course, the less active fraction of Python contributors may not
notice, since they just chose to not contribute (which, of course,
is fine). However, asking me to work twice as much as I want to
on the project to keep two branches alive is just unfair.

This has nothing to do with pushing 3.x, but all with managing
available manpower and still providing quality software.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 09:42:51 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 09:42:51 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>

This is what it says now:

"Python 2.7 is scheduled to be the last major version in the 2.x
series before it moves into 5 years of bugfix-only mode. "

And during this discussion it has been noted that others than the core
python team might pick up Python 2 and make releases. So maybe we can
and this discussion by changing that line in future releases to be:

"Python 2.7 is scheduled to be the last major version in the 2.x
series released by the Python Software Foundation before it moves into
5 years of bugfix-only mode. "

That doesn't exclude *others* making a Python 2.8 that could include
all sorts of crazy features. :)

This is mainly just to get the pointless discussion over with. The
current Python core team don't want to make a 2.8, so that will not
happen. Someone else might, but as Chris points out, they won't step
up until they have to, which is probably at least two years from now.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 10:06:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 10:06:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
Message-ID: <4B4AEA25.7010100@v.loewis.de>

> "Python 2.7 is scheduled to be the last major version in the 2.x
> series released by the Python Software Foundation before it moves into
> 5 years of bugfix-only mode. "
> 
> That doesn't exclude *others* making a Python 2.8 that could include
> all sorts of crazy features. :)
> 
> This is mainly just to get the pointless discussion over with.

I know you are just looking for a compromise, but this shouldn't be it:
the PSF has deliberately stayed out of the actual Python engineering,
so the release that Benjamin makes is not done by the PSF (but by
Benjamin and his contributors :-).

I like the wording as it is (although I find the promise of five years
of bug fix releases a bit too strong).

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 10:19:24 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 10:19:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4AEA25.7010100@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
Message-ID: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>

On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I know you are just looking for a compromise, but this shouldn't be it:
> the PSF has deliberately stayed out of the actual Python engineering,
> so the release that Benjamin makes is not done by the PSF (but by
> Benjamin and his contributors :-).

Hm. Yeah. That's right of course. I started with saying "official",
but then I thought "official according to who?". :) So I changed it to
mentioning PSF, but that doesn't work either. I guess the current
writing as as good as it gets, unless we change "scheduled" to
"expected" or something.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From stefan_ml at behnel.de  Mon Jan 11 10:42:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 Jan 2010 10:42:05 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111041754.GB7200@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A5C69.7090007@v.loewis.de> <20100111041754.GB7200@arctrix.com>
Message-ID: <hierpd$dm1$1@ger.gmane.org>

Neil Schemenauer, 11.01.2010 05:17:
> On Mon, Jan 11, 2010 at 12:02:01AM +0100, "Martin v. L?wis" wrote:
>> [...] would it still be ok if I closed a 2.x feature request as
>> "won't fix", or only committed the new feature to the 3.x branch?
> 
> Yes.  Non-bugfix development on 2.x would optional (i.e. done by
> people who want to spend the time).

Note that there's also the time it takes to make a release (usually 
involving at least one beta release), plus the possibility of introducing 
bugs while adding new features or even while fixing agreed bugs. All of 
this takes time in addition to the time people might want to invest in 
'real' development on the 2.x trunk.

Stefan



From stephen at xemacs.org  Mon Jan 11 10:59:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 11 Jan 2010 18:59:57 +0900
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111052713.GA8211@arctrix.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com>
Message-ID: <8763793qsi.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Schemenauer writes:
 > On Sun, Jan 10, 2010 at 09:06:15PM -0800, Brett Cannon wrote:

 > > If people start taking the carrots we have added to 3.x and
 > > backporting them to keep the 2.x series alive you are essentially
 > > making the 3.x DOA by negating its benefits which I personally
 > > don't agree with.

Well, I think it's *worse* than that, and I don't think you really
mean "DOA", anyway.  (Feel free to correct me, of course.)

The problem I see with backporting lots of stuff, and/or adding new
features that aren't in 3.0, to 2.x is that it will make 2.x even
cruftier, when it was already crufty enough that Guido (and almost all
of python-dev) bit the bullet and said "backward compatibility is no
excuse for keeping something in 3.0".

That surely means that a lot of python-dev denizens will declare
non-support 2.x for x > 7.  It's not going to be the gradual migration
we've seen over the past few months as active people start to spend
more and more time on 3 vs. 2; it will be a watershed.  Especially if
these are new features merged from outside that the "small active
segment" doesn't know anything about.  From the users' point of view,
that amounts to a *fork*, even if it's internal and "friendly".

 > I think we have got to the heart of our disagreement. Assume that
 > some superhuman takes all the backwards compatible goodies from 3.x
 > and merges them into 2.x.

Isn't that a bit ridiculous?  I just don't see any evidence that
anything like that is going to happen.  Worse, if we *assume* it will
happen, I don't see any way to assess whether (1) Python 3 goes belly
up, (2) there's an effective fork confusing the users and draining the
energy of python-dev, or (3) everybody goes "wow!" because it's so
cool that everybody wants to keep maintaining an extra 3 branches
indefinitely.

My opinion is that given the clear direction the "small active
segment" is going, telling the users anything but what Brett proposed
is disinformation.

 > I guess I have more confidence in Python 3 than you do. I don't see
 > why Python 2.x needs to be artificially limited so that Python 3 can
 > benefit.

It's not for Python 3, which you, I, and I'm pretty sure Brett-in-his-
heart-of-hearts agree can take care of itself because it is *better*
than Python 2.  It's for Python, and for the Python community.


From walter at livinglogic.de  Mon Jan 11 11:37:56 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 11:37:56 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <201001091438.43576.victor.stinner@haypocalc.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
Message-ID: <4B4AFF84.7070206@livinglogic.de>

On 09.01.10 14:38, Victor Stinner wrote:

> Le samedi 09 janvier 2010 12:18:33, Walter D?rwald a ?crit :
>>> Good idea, I choosed open(filename, encoding="BOM").
>>
>> On the surface this looks like there's an encoding named "BOM", but
>> looking at your patch I found that the check is still done in
>> TextIOWrapper. IMHO the best approach would to the implement a *real*
>> codec named "BOM" (or "sniff"). This doesn't require *any* changes to
>> the IO library. It could even be developed as a standalone project and
>> published in the Cheeseshop.
> 
> Why not, this is another solution to the point (2) (Check for a BOM while 
> reading or detect it before?). Which encoding would be used if there is not 
> BOM? UTF-8 sounds like a good choice.

UTF-8 might be a good choice, are the failback could be specified in the
encoding name, i.e.

   open("file.txt", encoding="BOM-UTF-8")

falls back to UTF-8, if there's no BOM at the start.

This could be implemented via a custom codec search function (see
http://docs.python.org/library/codecs.html#codecs.register for more info).

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 12:12:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:12:20 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4AFF84.7070206@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
Message-ID: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>

On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
> UTF-8 might be a good choice

No, fallback if there is no BOM should be the local settings, just as
fallback is today if you don't specify a codec.
I mean, what if you want to look for a BOM but fall back to something
else? How far will we go with encoding special information in the
codecs names? codec='BOM else UTF-16 else locale'? :-)

BOM is not a locale, and should not be a locale. Having a locale
called BOM is wrong per se. It should either be default to look for a
BOM when codec=None, or a special parameter. If none of these are
desired, then we need a special function that takes a filename or file
handle, and looks for a BOM and returns the codec found or None. But
I find that much less natural and obvious than checking the BOM when codec=None.

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From regebro at gmail.com  Mon Jan 11 12:13:00 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 12:13:00 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<201001091438.43576.victor.stinner@haypocalc.com>
	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <319e029f1001110313u1d839deodfe06a6c7ec423cf@mail.gmail.com>

On Mon, Jan 11, 2010 at 12:12, Lennart Regebro <regebro at gmail.com> wrote:
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se.

D'oh! I mean codec here obviously.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 13:29:04 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 13:29:04 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4913DF.7050801@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>
Message-ID: <4B4B1990.90705@livinglogic.de>

On 10.01.10 00:40, "Martin v. L?wis" wrote:
>>> How does the requirement that it be implemented as a codec miss the
>>> point?
>>
>> If we want it to be the default, it must be able to fallback on the current
>> locale-based algorithm if no BOM is found. I don't think it would be easy for a
>> codec to do that.
> 
> Yes - however, Victor currently apparently *doesn't* want it to be the
> default, but wants the user to specify encoding="BOM". If so, it isn't
> the default, and it is easy to implement as a codec.
> 
>>> FWIW, I agree with Walter that if it is provided through the encoding=
>>> argument, it should be a codec. If it is built into the open function
>>> (for whatever reason), it must be provided by some other parameter.
>>
>> Why not simply encoding=None?
> 
> I don't mind. Please re-read Walter's message - it only said that
> *if* this is activated through encoding="BOM", *then* it must be
> a codec, and could be on PyPI. I don't think Walter was talking about
> the case "it is not activated through encoding='BOM'" *at all*.

However if this autodetection feature is useful in other cases (no
matter how it's activated), it should be a codec, because as part of the
open() function it isn't reusable.

>> The default value should provide the most useful
>> behaviour possible. Forcing users to choose between two different autodetection
>> strategies (encoding=None and another one) is a little insane IMO.

And encoding="mbcs" is a third option on Windows.

> That wouldn't disturb me much. There are a lot of things in that area
> that are a little insane, starting with Microsoft Windows :-)

;)

Servus,
   Walter


From barry at python.org  Mon Jan 11 13:37:46 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 07:37:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4A5DCE.3070909@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com> <4B4A5DCE.3070909@v.loewis.de>
Message-ID: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>

On Jan 10, 2010, at 6:07 PM, Martin v. L?wis wrote:

> As for decisions: I don't think there was an official BDFL pronouncent,
> but I recall Guido posting a message close to that, proposing that 2.7
> will be a release that will see bug fix releases for an indefinite
> period of time (where indefinite != infinite). This was shortly after
> him proposing that perhaps we shouldn't make a 2.7 release at all, and
> stop at 2.6.

IMO, the focus for Python 2.7 (and beyond) must be on helping people transition to Python 3.  That should be the reason why a 2.7 is released and, what should dictate whether a 2.8 is necessary.

Maybe everything people need (except manpower and round tuits) is already there.  I was pleasantly surprised to find that Distribute supports automatically running 2to3 at install time for Python packages.  This allowed me to "port" a recent (admittedly small) package to Python 3 by making it "python2.6 -3" clean and adding a couple of attributes to my setup.py[1].  I don't know how widely that feature's known, but I think it's *huge*, and exactly what we need to be doing.  The more we can make it easy for people to port their Python 2 code to Python 3, and to support that with one code base, the quicker we'll get to a predominantly Python 3 world.

So the question we should be asking is: what's missing from Python 2.7 to help with this transition?  If we can't get it into 2.7, then why, and would pushing it back to 2.8 help at all?

-Barry

[1] modulo a bug in Distribute that caused doctest in separate files to not be used when running  under Python 3.



From solipsis at pitrou.net  Mon Jan 11 13:39:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 13:39:33 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de>  <4B4B1990.90705@livinglogic.de>
Message-ID: <1263213574.3327.0.camel@localhost>


> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

It is reusable as part of io.TextIOWrapper, though.

Regards

Antoine.




From regebro at gmail.com  Mon Jan 11 13:45:30 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 13:45:30 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B1990.90705@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<4B46F67F.60604@v.loewis.de>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
Message-ID: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>

On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
> However if this autodetection feature is useful in other cases (no
> matter how it's activated), it should be a codec, because as part of the
> open() function it isn't reusable.

But an autodetect feature is not a codec. Sure it should be reusable,
but making it a codec seems to be  a weird hack to me. And how would
you reuse it if it was a codec? A reusable autodetect feature would be
useable to detect what codec it is. A autodetect codec would not be
useful for that, as it would simply just decode.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From walter at livinglogic.de  Mon Jan 11 14:21:15 2010
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 11 Jan 2010 14:21:15 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<4B46F67F.60604@v.loewis.de>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<loom.20100109T160248-501@post.gmane.org>	<4B48E277.9010408@v.loewis.de>	<loom.20100109T212555-508@post.gmane.org>	<4B4913DF.7050801@v.loewis.de>
	<4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B25CB.5030003@livinglogic.de>

On 11.01.10 13:45, Lennart Regebro wrote:

> On Mon, Jan 11, 2010 at 13:29, Walter D?rwald <walter at livinglogic.de> wrote:
>> However if this autodetection feature is useful in other cases (no
>> matter how it's activated), it should be a codec, because as part of the
>> open() function it isn't reusable.
> 
> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

I think we already had this discussion two years ago in the context of
XML decoding ;):

http://mail.python.org/pipermail/python-dev/2007-November/075138.html

> And how would
> you reuse it if it was a codec? A reusable autodetect feature would be
> useable to detect what codec it is. A autodetect codec would not be
> useful for that, as it would simply just decode.

I have implemented an XML codec (as part of XIST:
http://pypi.python.org/pypi/ll-xist), that can do that:

>>> from ll import xml_codec
>>> import codecs
>>> c = codecs.getincrementaldecoder("xml")()
>>> c.encoding
>>> c.decode("<?xml")
u''
>>> c.encoding
>>> c.decode(" version='1.0'")
u''
>>> c.encoding
>>> c.decode(" encoding='iso-8859-1'?>")
u"<?xml version='1.0' encoding='iso-8859-1'?>"
>>> c.encoding
'iso-8859-1'

Servus,
   Walter


From regebro at gmail.com  Mon Jan 11 14:47:12 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 14:47:12 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B25CB.5030003@livinglogic.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B25CB.5030003@livinglogic.de>
Message-ID: <319e029f1001110547n1d1c9831p34b0eac519d13f9d@mail.gmail.com>

On Mon, Jan 11, 2010 at 14:21, Walter D?rwald <walter at livinglogic.de> wrote:
> I think we already had this discussion two years ago in the context of
> XML decoding ;):

Yup. Ans Martins answer then is my answer now:

"> So the code is good, if it is inside an XML parser, and it's bad if it
> is inside a codec?

Exactly so. This functionality just *isn't* a codec - there is no
encoding. Instead, it is an algorithm for *detecting* an encoding."

The conclusion was that a method do autodetect encodings would be
good. I think the same conclusion applies here.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Mon Jan 11 18:16:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:16:37 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<4B46F67F.60604@v.loewis.de>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
Message-ID: <4B4B5CF5.50806@v.loewis.de>

> But an autodetect feature is not a codec. Sure it should be reusable,
> but making it a codec seems to be  a weird hack to me.

Well, the existing UTF-16 codec also is an autodetect feature (to
detect the endianness), and I don't consider it a weird hack.

Regards,
Martin


From regebro at gmail.com  Mon Jan 11 18:27:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:27:01 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B5CF5.50806@v.loewis.de>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<201001081140.28124.victor.stinner@haypocalc.com>
	<4B486609.2050804@livinglogic.de>
	<loom.20100109T160248-501@post.gmane.org>
	<4B48E277.9010408@v.loewis.de>
	<loom.20100109T212555-508@post.gmane.org>
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>
	<4B4B5CF5.50806@v.loewis.de>
Message-ID: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:16, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> But an autodetect feature is not a codec. Sure it should be reusable,
>> but making it a codec seems to be ?a weird hack to me.
>
> Well, the existing UTF-16 codec also is an autodetect feature (to
> detect the endianness), and I don't consider it a weird hack.

So the BOM codec should raise a UnicodeDecodeError if there is no BOM?
Because that's what it would have to do, in that case, because it
can't fall back on anything, it has to handle and implement all
encodings that have a BOM. And is it then actually very useful? You
would have to do a try/except first with encoding='BOM' and then
encoding=None to get the fallback to the standard.


I must say that I find this whole thing pretty obvious. 'BOM' is not
an encoding. Either there should be a method to get the encoding from
the BOM, returning None of there isn't one, or open() should look at
the BOM when you pass in encoding=None. Or both.

That covers all usecases, is easy and obvious. Either open(file=foo,
encoding=None) or open(file, encoding=encoding_from_bom(file))

I can't see that open(file, encoding='BOM') has any benefit over this,
covers any extra usecase and is clearer in any way. Instead it adds
something confusing: An encoding that isn't an encoding.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From python at mrabarnett.plus.com  Mon Jan 11 18:35:35 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:35:35 +0000
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<201001081140.28124.victor.stinner@haypocalc.com>	<4B486609.2050804@livinglogic.de>	<201001091438.43576.victor.stinner@haypocalc.com>	<4B4AFF84.7070206@livinglogic.de>
	<319e029f1001110312t461dc126o1dcb5de381684e3@mail.gmail.com>
Message-ID: <4B4B6167.40606@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 11:37, Walter D?rwald <walter at livinglogic.de> wrote:
>> UTF-8 might be a good choice
> 
> No, fallback if there is no BOM should be the local settings, just as
> fallback is today if you don't specify a codec.
> I mean, what if you want to look for a BOM but fall back to something
> else? How far will we go with encoding special information in the
> codecs names? codec='BOM else UTF-16 else locale'? :-)
> 
> BOM is not a locale, and should not be a locale. Having a locale
> called BOM is wrong per se. It should either be default to look for a
> BOM when codec=None, or a special parameter. If none of these are
> desired, then we need a special function that takes a filename or file
> handle, and looks for a BOM and returns the codec found or None. But
> I find that much less natural and obvious than checking the BOM when codec=None.
> 
Or pass a function that accepts a byte stream or the first few bytes and
returns the encoding and any unused bytes (because the byte stream might
not be seekable)?

def guess_encoding(byte_stream):
     data = byte_stream.read(2)
     if data == b"\xFE\xFF":
         return "UTF-16BE", b""
     return "UTF-8", data

text_file = open(filename, encoding=guess_encoding)
...


From python at mrabarnett.plus.com  Mon Jan 11 18:46:30 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 11 Jan 2010 17:46:30 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
Message-ID: <4B4B63F6.1020309@mrabarnett.plus.com>

Lennart Regebro wrote:
> On Mon, Jan 11, 2010 at 10:06, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>> I know you are just looking for a compromise, but this shouldn't be
>> it: the PSF has deliberately stayed out of the actual Python
>> engineering, so the release that Benjamin makes is not done by the
>> PSF (but by Benjamin and his contributors :-).
> 
> Hm. Yeah. That's right of course. I started with saying "official", 
> but then I thought "official according to who?". :)

"Official" is whatever the BDFL says it is! :-)

> So I changed it to mentioning PSF, but that doesn't work either. I
> guess the current writing as as good as it gets, unless we change
> "scheduled" to "expected" or something.
> 


From martin at v.loewis.de  Mon Jan 11 18:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 18:59:08 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	
	<201001081140.28124.victor.stinner@haypocalc.com>	
	<4B486609.2050804@livinglogic.de>	
	<loom.20100109T160248-501@post.gmane.org>	
	<4B48E277.9010408@v.loewis.de>	
	<loom.20100109T212555-508@post.gmane.org>	
	<4B4913DF.7050801@v.loewis.de> <4B4B1990.90705@livinglogic.de>	
	<319e029f1001110445s6d892da3s226443383072a1b0@mail.gmail.com>	
	<4B4B5CF5.50806@v.loewis.de>
	<319e029f1001110927y36d82b3ftdf5ff56f24e57c4c@mail.gmail.com>
Message-ID: <4B4B66EC.7000203@v.loewis.de>

> I must say that I find this whole thing pretty obvious. 'BOM' is not
> an encoding.

That I certainly agree with.

> That covers all usecases, is easy and obvious. Either open(file=foo,
> encoding=None) or open(file, encoding=encoding_from_bom(file))
> 
> I can't see that open(file, encoding='BOM') has any benefit over this,

Well, it would have the advantage that Walter pointed out: you can
implement it independent of the open() implementation, and even provide
it in older versions of Python.

Regards,
Martin



From regebro at gmail.com  Mon Jan 11 18:59:52 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 11 Jan 2010 18:59:52 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B63F6.1020309@mrabarnett.plus.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
Message-ID: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>

On Mon, Jan 11, 2010 at 18:46, MRAB <python at mrabarnett.plus.com> wrote:
> "Official" is whatever the BDFL says it is! :-)

Heh, right.

So, it should say "Guido wants 2.7 to be the last main version of
Python 2, so it probably will be. We promise to release bugfixes it
for, like, ages".

No need to be needlessly formal. :-D
-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From olemis at gmail.com  Mon Jan 11 19:58:01 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 13:58:01 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
Message-ID: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>

> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>> BOM should be "ignored".
>>
[...]
>

I had similar issues too (please read below ;o) ...

On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
> talk. And for the other two, perhaps it would make more sense to have
> a separate encoding-guessing function that takes a binary stream and
> returns a text stream wrapping it with the proper encoding?
>

About guessing the encoding, I experienced this issue while I was
developing a Trac plugin. What I was doing is as follows :

- I guessed the MIME type + charset encoding using Trac MIME API (it
was a CSV file encoded using UTF-16)
- I read the file using `open`
- Then wrapped the file using `codecs.EncodedFile`
- Then used `csv.reader`

... and still get the BOM in the first value of the first row in the CSV file.

{{{
#!python

>>> mimetype
'utf-16-le'
>>> ef = EncodedFile(f, 'utf-8', mimetype)
}}}

IMO I think I am +1 for leaving `open` just like it is, and use module
`codecs` to deal with encodings, but I am strongly -1 for returning
the BOM while using `EncodedFile` (mainly because encoding is
explicitly supplied in ;o)

> --Guido
>

CMIIW anyway ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:


From mal at egenix.com  Mon Jan 11 21:44:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 11 Jan 2010 21:44:34 +0100
Subject: [Python-Dev] Improve open() to support reading file starting
 with an unicode BOM
In-Reply-To: <24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
Message-ID: <4B4B8DB2.1060102@egenix.com>

Olemis Lang wrote:
>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>> <victor.stinner at haypocalc.com> wrote:
>>> Hi,
>>>
>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>> BOM should be "ignored".
>>>
> [...]
>>
> 
> I had similar issues too (please read below ;o) ...
> 
> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>> talk. And for the other two, perhaps it would make more sense to have
>> a separate encoding-guessing function that takes a binary stream and
>> returns a text stream wrapping it with the proper encoding?
>>
> 
> About guessing the encoding, I experienced this issue while I was
> developing a Trac plugin. What I was doing is as follows :
> 
> - I guessed the MIME type + charset encoding using Trac MIME API (it
> was a CSV file encoded using UTF-16)
> - I read the file using `open`
> - Then wrapped the file using `codecs.EncodedFile`
> - Then used `csv.reader`
> 
> ... and still get the BOM in the first value of the first row in the CSV file.

You didn't say, but I presume that the charset guessing logic
returned either 'utf-16-le' or 'utf-16-be' - those encodings don't
remove the leading BOM. The 'utf-16' codec will remove the BOM.

> {{{
> #!python
> 
>>>> mimetype
> 'utf-16-le'
>>>> ef = EncodedFile(f, 'utf-8', mimetype)
> }}}

Same here: the UTF-8 codec will not remove the BOM, you have
to use the 'utf-8-sig' codec for that.

> IMO I think I am +1 for leaving `open` just like it is, and use module
> `codecs` to deal with encodings, but I am strongly -1 for returning
> the BOM while using `EncodedFile` (mainly because encoding is
> explicitly supplied in ;o)

Note that EncodedFile() doesn't do any fancy BOM detection or
filtering. This is the job of the codecs.

Also note that BOM removal is only valid at the beginning of
a file. All subsequent BOM-bytes have to be read as-is (they
map to a zero-width non-breaking space) - without removing them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 11 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Mon Jan 11 22:12:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 07:12:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
Message-ID: <4B4B9447.4060101@gmail.com>

Benjamin Peterson wrote:
 > My question is: Is this a doc bug or a implementation bug? If the
> former, it will be the description of a data descriptor much less
> consistent, since it will require that a __get__ method be present,
> too. If the latter, the fix may break some programs relying on the
> ability to "cache" a value in the instance dictionary.
> 
> [1] http://docs.python.org/reference/datamodel#invoking-descriptors

I would call it a documentation bug: by leaving out the __get__ you're
telling Python to override *just* the setting of the attribute, while
still using the normal lookup process for retrieving the attribute (i.e.
allowing the attribute to be shadowed in the instance dictionary).

Adding a "print('Descriptor Invoked')" to the __set__ method in your
example:

>>> x = X()
>>> print(x.attr)
<__main__.Descr object at 0x7f283b51ac50>
>>> x.attr = 42
Descriptor invoked
>>> print(x.attr)
42
>>> x.attr = 6*9
Descriptor invoked
>>> print(x.attr)
54

Note that the behaviour here is still different from that of a data
descriptor: with a data descriptor, once it gets shadowed in the
instance dictionary, the descriptor is ignored *completely*. The only
way to get the descriptor involved again is to eliminate the shadowing.
The non-data descriptor with only __set__ is just choosing not to
override the attribute lookup process.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From olemis at gmail.com  Mon Jan 11 22:29:38 2010
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 11 Jan 2010 16:29:38 -0500
Subject: [Python-Dev] Improve open() to support reading file starting
	with an unicode BOM
In-Reply-To: <4B4B8DB2.1060102@egenix.com>
References: <201001080110.35800.victor.stinner@haypocalc.com>
	<ca471dc21001071652n7d5746e1q87f064da74395c01@mail.gmail.com>
	<24ea26601001111058g7f9da075m70e765d6ba8b2086@mail.gmail.com>
	<4B4B8DB2.1060102@egenix.com>
Message-ID: <24ea26601001111329i7f609aa1i9f5db7eb61f1fae7@mail.gmail.com>

Probably one part of this is OT , but I think it could complement the
discussion ;o)

On Mon, Jan 11, 2010 at 3:44 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Olemis Lang wrote:
>>> On Thu, Jan 7, 2010 at 4:10 PM, Victor Stinner
>>> <victor.stinner at haypocalc.com> wrote:
>>>> Hi,
>>>>
>>>> Builtin open() function is unable to open an UTF-16/32 file starting with a
>>>> BOM if the encoding is not specified (raise an unicode error). For an UTF-8
>>>> file starting with a BOM, read()/readline() returns also the BOM whereas the
>>>> BOM should be "ignored".
>>>>
>> [...]
>>>
>>
>> I had similar issues too (please read below ;o) ...
>>
>> On Thu, Jan 7, 2010 at 7:52 PM, Guido van Rossum <guido at python.org> wrote:
>>> I'm a little hesitant about this. First of all, UTF-8 + BOM is crazy
>>> talk. And for the other two, perhaps it would make more sense to have
>>> a separate encoding-guessing function that takes a binary stream and
>>> returns a text stream wrapping it with the proper encoding?
>>>
>>
>> About guessing the encoding, I experienced this issue while I was
>> developing a Trac plugin. What I was doing is as follows :
>>
>> - I guessed the MIME type + charset encoding using Trac MIME API (it
>> was a CSV file encoded using UTF-16)
>> - I read the file using `open`
>> - Then wrapped the file using `codecs.EncodedFile`
>> - Then used `csv.reader`
>>
>> ... and still get the BOM in the first value of the first row in the CSV file.
>
> You didn't say, but I presume that the charset guessing logic
> returned either 'utf-16-le' or 'utf-16-be'

Yes. In fact they return the full mimetype 'text/csv; charset=utf-16-le' ... ;o)

> - those encodings don't
> remove the leading BOM.

... and they should ?

> The 'utf-16' codec will remove the BOM.
>

In this particular case there's nothing I can do, I have to process
whatever charset is detected in the input ;o)

>> {{{
>> #!python
>>
>>>>> mimetype
>> 'utf-16-le'
>>>>> ef = EncodedFile(f, 'utf-8', mimetype)
>> }}}
>
> Same here: the UTF-8 codec will not remove the BOM, you have
> to use the 'utf-8-sig' codec for that.
>
>> IMO I think I am +1 for leaving `open` just like it is, and use module
>> `codecs` to deal with encodings, but I am strongly -1 for returning
>> the BOM while using `EncodedFile` (mainly because encoding is
>> explicitly supplied in ;o)
>
> Note that EncodedFile() doesn't do any fancy BOM detection or
> filtering.

... directly.

> This is the job of the codecs.
>

OK ... to come back to the scope of the subject, in the general case,
I think that BOM (if any) should be handled by codecs, and therefore
indirectly by EncodedFile . If that's a explicit way of working with
encodings I'd prefer to use that wrapper explicitly in order to
(encode | decode) the file and let the codec detect whether there's a
BOM or not and ?adjust? `tell`, `read` and everything else in that
wrapper (instead of `open`).

> Also note that BOM removal is only valid at the beginning of
> a file. All subsequent BOM-bytes have to be read as-is (they
> map to a zero-width non-breaking space) - without removing them.
>

;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Test cases for custom query (i.e report 9) ... PASS (1.0.0)  -
http://simelo.hg.sourceforge.net/hgweb/simelo/trac-gviz/rev/d276011e7297


From martin at v.loewis.de  Mon Jan 11 22:42:45 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:42:45 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
Message-ID: <4B4B9B55.1010709@v.loewis.de>

> So the question we should be asking is: what's missing from Python
> 2.7 to help with this transition?

Wrt. the distribute feature you've noticed: Python 3.0 already supports
that in distutils. So from day one of Python 3.x, you could have used
that approach for porting Python code. I had been promoting it ever
since, but it's taking up only slowly, partially because it has
competition in the approaches "burn the bridges" (i.e. convert to 3.x
once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
(i.e. try to write code that works in both 2.x and 3.x unmodified).

> If we can't get it into 2.7, then
> why, and would pushing it back to 2.8 help at all?

I've done a fair bit of 3.x porting, and I'm firmly convinced that
2.x can do nothing:

a) telling people that they have to move to 2.6 first actually
   hurts migration, instead of helping, because it implies to them
   that they have to drop old versions (e.g. 2.3.) - just because
   they had *always* dropped old versions before supporting new ones.
b) IMO, people also don't gain much by first migrating to 2.6.
   In principle, it gives them the opportunity to get py3k warnings.
   However, I haven't heard a single positive report where these
   warnings have actually helped people in porting. Yours is the
   first report saying that you followed the official guideline,
   but you didn't state whether doing so actually helped (or whether
   you just ported to 2.6 because the guideline told you to).
c) whatever 2.7 does (except perhaps for the warnings), it won't
   be useful to applications, because they couldn't use it, anyway:
   they need to support 2.4 and 2.5, and it won't have any of the
   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
   might actually hurt porting: people may have to special-case
   2.7 because their work-arounds for older versions may break in 2.7
   (e.g. testing whether a name is *not* defined, when it becomes
   defined in 2.7), plus it gives them an incentive to not port
   yet until they can drop support for anything before 2.7.

Inherently, 2.8 can't improve on that.

Regards,
Martin


From martin at v.loewis.de  Mon Jan 11 22:44:24 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 11 Jan 2010 22:44:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
Message-ID: <4B4B9BB8.3070505@v.loewis.de>

> So, it should say "Guido wants 2.7 to be the last main version of
> Python 2, so it probably will be. We promise to release bugfixes it
> for, like, ages".
> 
> No need to be needlessly formal. :-D

That summarizes my understanding of what Guido said fairly well :-)

+1 for putting it into the announcements.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Mon Jan 11 23:11:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 11 Jan 2010 22:11:44 +0000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4B9447.4060101@gmail.com>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com>
Message-ID: <4B4BA220.20701@voidspace.org.uk>

On 11/01/2010 21:12, Nick Coghlan wrote:
> Benjamin Peterson wrote:
>   >  My question is: Is this a doc bug or a implementation bug? If the
>    
>> former, it will be the description of a data descriptor much less
>> consistent, since it will require that a __get__ method be present,
>> too. If the latter, the fix may break some programs relying on the
>> ability to "cache" a value in the instance dictionary.
>>
>> [1] http://docs.python.org/reference/datamodel#invoking-descriptors
>>      
> [snip...]
>
> Note that the behaviour here is still different from that of a data
> descriptor: with a data descriptor, once it gets shadowed in the
> instance dictionary, the descriptor is ignored *completely*. The only
> way to get the descriptor involved again is to eliminate the shadowing.
> The non-data descriptor with only __set__ is just choosing not to
> override the attribute lookup process.
>
>    

Does that mean we need a third class of descriptors that are neither 
data descriptors nor non-data descriptors?

What should we call them: really-not-data-descriptors?

All the best,

Michael

> Cheers,
> Nick.
>
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From guido at python.org  Mon Jan 11 23:55:01 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 11 Jan 2010 14:55:01 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9BB8.3070505@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com> 
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de> 
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com> 
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com> 
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com> 
	<4B4B9BB8.3070505@v.loewis.de>
Message-ID: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>

+1 from me. (With the caveat that "time will tell" is definitely the
ultimate answer. Plans may change unexpectedly, even though we don't
expect them to.)

PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
helpful. But I trust he has ported a lot more code to 3.x than I have
personally. Are there other experiences?

--Guido

On Mon, Jan 11, 2010 at 1:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So, it should say "Guido wants 2.7 to be the last main version of
>> Python 2, so it probably will be. We promise to release bugfixes it
>> for, like, ages".
>>
>> No need to be needlessly formal. :-D
>
> That summarizes my understanding of what Guido said fairly well :-)
>
> +1 for putting it into the announcements.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Tue Jan 12 00:16:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 00:16:47 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <4B4BB15F.5020807@v.loewis.de>

Guido van Rossum wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
> 
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. 

That's really because I haven't even attempted to use it. Instead, the
software would fall flat on its own when running the test suite in 3.x.
IOW, I don't need 2.6 to tell me what might break when I can see 3.x
actually breaking.

Regards,
Martin


From david.lyon at preisshare.net  Tue Jan 12 00:22:42 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:22:42 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4ADED6.5080207@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
Message-ID: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>


Hi Martin,

> Of course, the less active fraction of Python contributors may not
> notice, since they just chose to not contribute (which, of course,
> is fine). However, asking me to work twice as much as I want to
> on the project to keep two branches alive is just unfair.

Totally true. Actually as an end-developer I'd say that python
2.x series from a programming perspective is quite good. It
doesn't need the addition of a lot of new features imho.

So for me, I think too much time spent there would be not
yield great benefits.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Well, we all know your work is super quality. :-) That's not
being contested.

However, Quality can be measured different ways and it can
be assessed in different ways. Quality itself is a subjective
thing.

The point I'm only making is that if a piece of software
doesn't have "new" things added over time, then users
can get a reverse impression of a lack of quality.

We've all seen where 'internal' quality can increase
and user perceptions can decrease.

It could be things like improved graphics and things readily
apparent to the user.

At the moment, I would say that the "internal" quality
of the python 2.x series is super high. "external" quality
issues such as the packaging dilemma give the user the
opposite quality experience. But people are working on
that as best they can elsewhere. I'll leave it at that.

> This has nothing to do with pushing 3.x, but all with managing
> available manpower and still providing quality software.

Python 3.x needs more carrots.

>From an ordinary (perphaps ignorant) user perspective there
is nothing. Yes, we know if we actually will start
programming then we will like it more.

But my wishes to Santa Claus would be allow the free
flow of PEPs for Python 3 packaging. Even encourage
it.

As an end developer, here's what I'd like from Santa
in 2010 to get me to swap to python 3:

 * get all the packages on pypi tested for python 3

 * put a web based package manager in python 3. This
   would perhaps be based around PIP (keep many
   people happy) and would look much like the built
   in web-console that you get with a $200 router.

 * Incorporate SCM based end-user software installs
   by adding to python3-stdlib the SCM support
   packages (cvs,bzr,svn,hg). This would *really*
   help in the scientific community.

 * put a web interface on distutils also so that
   we don't have to use a command line interface.
   (I want a pic of a smiley girl to great me
    when I build something - "Are you ready
    to build now Honey?"). ok - I joke. But the
    point is made.

So, ok, maybe these things aren't about 'code'
quality. But rather user experience.

Things like these do count also as "quality"
via the technical term "perception of quality".

If the PEP process is as unblocked as the
documentation implies, implying that anybody
can contribute to Python 3. Then there shouldn't
be any issue.

David









From solipsis at pitrou.net  Tue Jan 12 00:37:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Jan 2010 23:37:47 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100112T003506-611@post.gmane.org>

David Lyon <david.lyon <at> preisshare.net> writes:
> 
> > This has nothing to do with pushing 3.x, but all with managing
> > available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As someone who experiences the difference almost every day, I can say 3.x
definitely has enough carrots for me. The simple fact that I will be able to
raise exceptions with non-ASCII error messages without having to workaround a
hopelessly broken conversion/reporting logic is a huge improvement. And that's
only a small part of the picture.

Regards

Antoine.




From janssen at parc.com  Tue Jan 12 00:47:55 2010
From: janssen at parc.com (Bill Janssen)
Date: Mon, 11 Jan 2010 15:47:55 PST
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <17215.1263253675@parc.com>

> David Lyon <david.lyon <at> preisshare.net> writes:
> > 
> > > This has nothing to do with pushing 3.x, but all with managing
> > > available manpower and still providing quality software.
> > 
> > Python 3.x needs more carrots.

I'd be happy to move UpLib to Python 3, when the various packages that I
need are also on Python 3.  And that depresses me to think about.  I
need

   Medusa
   ReportLab
   PyLucene
   Email
   Vobject
   Mutagen
   Pyglet
   Hachoir
   Win32

I'm on the mailing lists for a lot of these, and the only one that I
know is thinking about Python 3 is Email.  I'd expect Win32 and PIL to
also be thinking about it, but I haven't heard anything.

Bill


From david.lyon at preisshare.net  Tue Jan 12 00:47:51 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 12 Jan 2010 10:47:51 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <loom.20100112T003506-611@post.gmane.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<loom.20100112T003506-611@post.gmane.org>
Message-ID: <1220.218.214.45.58.1263253671.squirrel@syd-srv02.ezyreg.com>

Antoine writes:

> As someone who experiences the difference almost every day, I can say 3.x
> definitely has enough carrots for me. The simple fact that I will be able
> to
> raise exceptions with non-ASCII error messages without having to
> workaround a
> hopelessly broken conversion/reporting logic is a huge improvement. And
> that's
> only a small part of the picture.

In no way could I disagree with you.

Ascii works fine for us here - but I guess we're lucky.

Just keep pushing python 3 and have a nice day..

David





From barry at python.org  Tue Jan 12 01:11:28 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:11:28 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100111191128.39020d89@freewill>

On Jan 11, 2010, at 10:42 PM, Martin v. L?wis wrote:

>Wrt. the distribute feature you've noticed: Python 3.0 already supports
>that in distutils. So from day one of Python 3.x, you could have used
>that approach for porting Python code. I had been promoting it ever
>since, but it's taking up only slowly, partially because it has
>competition in the approaches "burn the bridges" (i.e. convert to 3.x
>once, and then have two code bases, or abandon 2.x), and "avoid 2to3"
>(i.e. try to write code that works in both 2.x and 3.x unmodified).

Interesting.  The only reason I never used it before is because I didn't know
about it.  Am I alone?

Maybe I'm projecting my own preferences, but it seems to me that supporting
both Python 2 and 3 from the same code base would be a very powerful way to
enable a large amount of existing code to support Python 3.  I'm certainly
going to do this from now on with all the libraries I maintain.  I don't have
any cycles to write code twice and I can't abandon Python 2 yet.  I'm
skeptical that code can work unmodified in both 2 and 3 without 2to3.

As an example, the one library I've already ported used a metaclass.  I don't
see any way to specify that the metaclass should be used in a portable way.
In Python 2.6 it's:

class Foo:
    __metaclass__ = Meta

and in Python 3 it's:

class Foo(metaclass=Meta):

2to3 made that pain go away.

>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>2.x can do nothing:
>
>a) telling people that they have to move to 2.6 first actually
>   hurts migration, instead of helping, because it implies to them
>   that they have to drop old versions (e.g. 2.3.) - just because
>   they had *always* dropped old versions before supporting new ones.

Is it just an implication, or is it reality?  I have yet to try to support
older versions of Python and also support Python 3.  (The one package I've
done this with so far only supports Python 2.6.)

>b) IMO, people also don't gain much by first migrating to 2.6.
>   In principle, it gives them the opportunity to get py3k warnings.
>   However, I haven't heard a single positive report where these
>   warnings have actually helped people in porting. Yours is the
>   first report saying that you followed the official guideline,
>   but you didn't state whether doing so actually helped (or whether
>   you just ported to 2.6 because the guideline told you to).

Python 2.6 has other useful features, which I want to take advantage of, so
getting py3k warnings is a bonus.  Running 'python2.6 -3 setup.py test' over
my code did in fact help clean up a couple of problems.  Seems like a good
first step when considering Python 3 support to me.

>c) whatever 2.7 does (except perhaps for the warnings), it won't
>   be useful to applications, because they couldn't use it, anyway:
>   they need to support 2.4 and 2.5, and it won't have any of the
>   gimmicks people come up with for 2.7. Adding gimmicks to 2.7
>   might actually hurt porting: people may have to special-case
>   2.7 because their work-arounds for older versions may break in 2.7
>   (e.g. testing whether a name is *not* defined, when it becomes
>   defined in 2.7), plus it gives them an incentive to not port
>   yet until they can drop support for anything before 2.7.
>
>Inherently, 2.8 can't improve on that.

I'm not so sure.  Yes, as a package maintainer there are older versions to
think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
released, what will be the minimum version of Python provided by most OS
vendors (where the majority of Python users probably get their 'python')?  I
guess some people will have to support everything from Python 2.3 to 2.8 but
you're talking supporting something like a spread of 7 years of Python
versions.  What other platform do you support for 7 years?  Even Ubuntu long
term support (LTS) releases only live for 5 years and even then, newer Pythons
are often back ported.  Or if not, then who cares?  You're not going to use a
newer version of a library on an LTS anyway.

I'm not necessarily saying that justifies a 2.8 release.  I'm just asking how
we can make it easier for people to port to Python 3.  The automatic 2to3 has
already made it easier for me to port one library to Python 3 because I barely
had to even think about it.  Maybe that's enough.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/840169eb/attachment-0007.pgp>

From barry at python.org  Tue Jan 12 01:12:19 2010
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Jan 2010 19:12:19 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <20100111191219.5fdd2542@freewill>

On Jan 11, 2010, at 02:55 PM, Guido van Rossum wrote:

>PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
>helpful. But I trust he has ported a lot more code to 3.x than I have
>personally. Are there other experiences?

I've only done one small library so far, but it was helpful to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/ba78a57a/attachment-0007.pgp>

From andrew at bemusement.org  Tue Jan 12 01:07:26 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Tue, 12 Jan 2010 11:07:26 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4B9B55.1010709@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
Message-ID: <20100112000726.GO32351@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> I've done a fair bit of 3.x porting, and I'm firmly convinced that
> 2.x can do nothing:
[...]
> Inherently, 2.8 can't improve on that.

I agree that there are limitations like the ones you've listed, but I
disagree with your conclusion.  Maybe you assume that it's just as hard
to move to 2.8 (using the py3k backports not available in say 2.5) as it
is to 3.x?

But a hypothetical 2.8 would also give people a way to move closer to
py3k without giving up on using all their 2.x-only dependencies.  I
think it's much more likely that libraries like Twisted can support 2.8
in the near future than 3.x.

Then, when all of your dependencies (or viable alternatives to those
dependencies) are available for 3.x, you'll have an easier transition if
you can start from a 2.x with fewer differences in features.

Fundamentally the more 2.x can converge on 3.x, the easier it will be
for users to make the leap, because it will be a smaller leap.  The
longer the 2.x series lives, the more these newer 2.x versions like 2.7
and maybe even 2.8 will be available on common platforms for people to
depend upon as minimum versions, which means that as time goes by they
can depend on a version that's closer to 3.x.  And so again, the leap
becomes easier to make.  So to me it's pretty clear that 2.8 *can*
improve the transition path to 3.x.  It may or may not be a worthwhile
use of effort for python-dev to make 2.8, but that's different to saying
it's inherently pointless.

-Andrew.


From solipsis at pitrou.net  Tue Jan 12 01:28:26 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 00:28:26 +0000 (UTC)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <loom.20100112T012550-387@post.gmane.org>

Andrew Bennetts <andrew <at> bemusement.org> writes:
> 
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies.  I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

I don't think a 2.8 could make you much closer to 3.x. AFAICT, every possible
way to ease transition to 3.x will already be in 2.7, or almost. But perhaps I'm
lacking imagination.

Regards

Antoine.




From brian.curtin at gmail.com  Tue Jan 12 02:57:46 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 11 Jan 2010 19:57:46 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>

On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:

> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
>
> -Brett
>
I think it would be interesting to see how people are using the tracker, or
how they want to be using it. For example, there are currently over 1500
open issues with no stage set, some of which seemingly haven't been read by
anyone at all. Would a properly set stage field save issues from falling
into a black hole?

Food for thought: according to the last tracker summary, there are over 1000
open issues with a patch, and issues stay open an average of 700 days
(median 450).

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100111/a3439436/attachment-0007.htm>

From jackdied at gmail.com  Tue Jan 12 04:53:24 2010
From: jackdied at gmail.com (Jack Diederich)
Date: Mon, 11 Jan 2010 22:53:24 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>

On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

[sidebar]
1) the metaclass fixer was a PITA to implement.
2) 95% of __metaclass__ definitions searchable via google code were of
the "__metaclass__ = type" variety.  The 2to3 patch exists only
because of the few other uses.
3) 100% of the module level assignments in public projects were the
"__metaclass__ = type" variety which is why there isn't a fixer for
that.  Also, a fixer would have been really, really ugly (munge every
class definition in this module because there is a top level
assignment).

-Jack


From benjamin at python.org  Tue Jan 12 05:01:40 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 Jan 2010 22:01:40 -0600
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <1afaf6161001112001t5e7bab83t7d93f33fa11ce849@mail.gmail.com>

2010/1/11 Jack Diederich <jackdied at gmail.com>:
> On Mon, Jan 11, 2010 at 7:11 PM, Barry Warsaw <barry at python.org> wrote:
>> As an example, the one library I've already ported used a metaclass. ?I don't
>> see any way to specify that the metaclass should be used in a portable way.
>> In Python 2.6 it's:
>>
>> class Foo:
>> ? ?__metaclass__ = Meta
>>
>> and in Python 3 it's:
>>
>> class Foo(metaclass=Meta):
>>
>> 2to3 made that pain go away.
>
> [sidebar]
> 1) the metaclass fixer was a PITA to implement.

Does this make it any less useful, though? :)



-- 
Regards,
Benjamin


From steven.bethard at gmail.com  Tue Jan 12 06:57:18 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 11 Jan 2010 21:57:18 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>

On Mon, Jan 11, 2010 at 4:11 PM, Barry Warsaw <barry at python.org> wrote:
> As an example, the one library I've already ported used a metaclass. ?I don't
> see any way to specify that the metaclass should be used in a portable way.
> In Python 2.6 it's:
>
> class Foo:
> ? ?__metaclass__ = Meta
>
> and in Python 3 it's:
>
> class Foo(metaclass=Meta):
>
> 2to3 made that pain go away.

Actually there's a solution to this one too:

    FooBase = Meta('FooBase', (), {})
    class Foo(FooBase):
        ...

That should work in Python 2.X and 3.X.

I've got argparse running on Python 2.3-3.1, and the changes were
pretty easy. You can see them all in the revision here:

    http://code.google.com/p/argparse/source/detail?r=12

I have aspirations of putting all of the tricks I learned up up on the
Wiki somewhere, but I just haven't had the time.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From regebro at gmail.com  Tue Jan 12 07:30:10 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:30:10 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>
	<4B4AEA25.7010100@v.loewis.de>
	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>
	<4B4B63F6.1020309@mrabarnett.plus.com>
	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>
	<4B4B9BB8.3070505@v.loewis.de>
	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
Message-ID: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>

On Mon, Jan 11, 2010 at 23:55, Guido van Rossum <guido at python.org> wrote:
> +1 from me. (With the caveat that "time will tell" is definitely the
> ultimate answer. Plans may change unexpectedly, even though we don't
> expect them to.)
>
> PS. I'm surprised that Martin thinks that the -3 mode in 2.6 is not
> helpful. But I trust he has ported a lot more code to 3.x than I have
> personally. Are there other experiences?

It doesn't warn for that many of the unportable problems, but I'm not
sure it can warn for them either. It's certainly helpful, just not
very much. :) I think the biggest help was added in 2.6.2, and that's
warning for old integer division. It will also warn for modules that
aren't supported anymore, if I remember correctly, and that's often
helpful. But it won't warn for real tricky problems, like binary vs
text in strings, and I don't see how it can either.

And, I just realized, it doesn't warn for you using cmp or __cmp__
either, and 2to3 won't fix that, so it should actually warn for it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:49:27 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:49:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
Message-ID: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:11, Barry Warsaw <barry at python.org> wrote:
> Interesting. ?The only reason I never used it before is because I didn't know
> about it. ?Am I alone?

Maybe not, but the Distribute feature is there because IMO the
distutils feature by itself isn't particularily useful. You need to
write your own distutils extensions, in practice, and they are not
trivial. Distribute has simply done it for you. :)

> I'm skeptical that code can work unmodified in both 2 and 3 without 2to3.
-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From regebro at gmail.com  Tue Jan 12 07:53:20 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 07:53:20 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <319e029f1001112253x511f8109ja2300a022010ef99@mail.gmail.com>

On Tue, Jan 12, 2010 at 01:07, Andrew Bennetts <andrew at bemusement.org> wrote:
> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. ?I
> think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

When 2.7 was discussed several people agreed that 2.7 should include
as much 3.x stuff as possible to ease transition. That turned out to
not be very much, so I'm not sure there is more. :)

Unless of course, 2.8 starts including more of the refactored
libraries, but that's a very minor issue in porting, so it won't help
much.

To really help, it needs to start implement things that break
backwards compatibility. That would be weird. :)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ncoghlan at gmail.com  Tue Jan 12 10:39:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:39:39 +1000
Subject: [Python-Dev] Data descriptor doc/implementation inconsistency
In-Reply-To: <4B4BA220.20701@voidspace.org.uk>
References: <1afaf6161001101607l19860878k7edc431510e2caba@mail.gmail.com>
	<4B4B9447.4060101@gmail.com> <4B4BA220.20701@voidspace.org.uk>
Message-ID: <4B4C435B.7080903@gmail.com>

Michael Foord wrote:
>> Note that the behaviour here is still different from that of a data
>> descriptor: with a data descriptor, once it gets shadowed in the
>> instance dictionary, the descriptor is ignored *completely*. The only
>> way to get the descriptor involved again is to eliminate the shadowing.
>> The non-data descriptor with only __set__ is just choosing not to
>> override the attribute lookup process.
>>
>>    
> 
> Does that mean we need a third class of descriptors that are neither
> data descriptors nor non-data descriptors?

Not really - leaving out __get__ when defining __set__ just creates a
data descriptor that happens to use the default attribute look-up
process rather than defining a different one.

(Note that I had the data/non-data terminology backwards in my previous
message - I tend to think of the two kinds of descriptor as enforced and
non-enforced respectively precisely because I have to think about it to
remember that "functions are non-data descriptors and properties are
data descriptors, hence non-data descriptors only define __get__ while
data descriptors define __set__". I don't find the data/non-data
terminology intuitive at all)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:44:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:44:18 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>	<319e029f1001110042ybc57c62w25e380707cd3ae48@mail.gmail.com>	<4B4AEA25.7010100@v.loewis.de>	<319e029f1001110119x39695088yf85c6ec64b6eba1e@mail.gmail.com>	<4B4B63F6.1020309@mrabarnett.plus.com>	<319e029f1001110959g68a9f1d9xe40e51f88d6db244@mail.gmail.com>	<4B4B9BB8.3070505@v.loewis.de>	<ca471dc21001111455g514811d4pa70dfa90d7f13fe@mail.gmail.com>
	<319e029f1001112230i3251a1c1k2ab7d159ce755f75@mail.gmail.com>
Message-ID: <4B4C4472.10104@gmail.com>

Lennart Regebro wrote:
> And, I just realized, it doesn't warn for you using cmp or __cmp__
> either, and 2to3 won't fix that, so it should actually warn for it.

I have a vague recollection that we tried to warn for that and ended up
nixing the warning due to vast swarms of false alarms (or because you
couldn't write correct 2.x code without triggering it).

I'd be happy for someone to prove my recollection wrong though (i.e. by
writing a patch that implements such warnings).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Tue Jan 12 10:48:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 19:48:57 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>	<20100111014457.GA5289@arctrix.com>	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>	<20100111040507.GA7200@arctrix.com>	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>	<20100111052713.GA8211@arctrix.com>
	<4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B4C4589.70906@gmail.com>

David Lyon wrote:
>> This has nothing to do with pushing 3.x, but all with managing
>> available manpower and still providing quality software.
> 
> Python 3.x needs more carrots.

As Guido has said a few times, the gains are far greater for *new*
Python developers than they are for existing ones.

Existing developers have to unlearn old habits and wait for libraries
they already use to get ported. New developers can just get started with
a much cleaner language. They don't have as rich a 3rd party ecosystem
on Python 3 as they would on Python 2.x at this point in time, but
unlike existing developers they also have the luxury of cherry-picking
just those packages that already have Python 3 support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Tue Jan 12 11:20:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:20:27 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <hihida$unc$1@ger.gmane.org>

Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit?:
> 
> For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all.

I think most issues /have/ been read. It's just that for many of them, 
nobody is interested enough in or feels competent enough for fixing them.

> Would a properly set stage field save issues from
> falling into a black hole?

I don't think this has anything to do with properly setting the stage 
field. We just have limited time and manpower. Perhaps one of our goals 
should be to reach out more to potential contributors.

> Food for thought: according to the last tracker summary, there are over
> 1000 open issues with a patch, and issues stay open an average of 700
> days (median 450).

As for open issues with a patch, a "code review" button sending this 
patch to Rietveld (or any other similar tool) is something many of us 
have been hoping for :-) It would make reviewing easier, faster and more 
thorough (because you visualize things better than by looking at a diff).

Regards

Antoine.



From solipsis at pitrou.net  Tue Jan 12 11:36:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Jan 2010 10:36:49 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <hihjc1$45m$1@ger.gmane.org>

Le Tue, 12 Jan 2010 10:20:27 +0000, Antoine Pitrou a ?crit?:
> 
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.

Speaking of which, Steve had something to say about it:
http://holdenweb.blogspot.com/2009/11/comments-or-not-public-or-
private.html

cheers

Antoine.



From ncoghlan at gmail.com  Tue Jan 12 13:10:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 12 Jan 2010 22:10:14 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <4B4C66A6.2040601@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
>> For example, there are currently over
>> 1500 open issues with no stage set, some of which seemingly haven't been
>> read by anyone at all.
> 
> I think most issues /have/ been read. It's just that for many of them, 
> nobody is interested enough in or feels competent enough for fixing them.

There are actually a whole host of reasons issues can stagnate:
- a feature request may seem reasonable (hence it doesn't get rejected
outright), but the right API may not be clear (hence it doesn't get
implemented in the near term)
- a patch may be reviewed and found to have significant defects (or
simply be overreaching the stated goal) but the original patch poster
loses interest after meeting resistance in their ambition to fix
something that is "obviously" broken
- a problem may simply be hard to fix in a backwards compatible way (or
even at all!)

Ultimately it boils down to Antoine's two categories (lack of interest
and lack of relevant expertise to make a final decision) but there are a
lot of different ways to get into those two buckets.

Because we aren't ruthless about pruning those kinds of issues out of
the tracker they're the ones that are going to accumulate over time.

I'd actually be interested to know what the issue stats are like when
RFEs are excluded and when the search is limited to the items flagged as
'easy'. If easy bug fix issues are taking a long time to get closed than
that would bother me a lot more than RFEs that sit on the tracker for years.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 12 13:14:47 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:14:47 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<b8e622741001111953k61ceb1c6kf36d20abe0bb6211@mail.gmail.com>
Message-ID: <20100112071447.675c8f24@freewill>

On Jan 11, 2010, at 10:53 PM, Jack Diederich wrote:

>3) 100% of the module level assignments in public projects were the
>"__metaclass__ = type" variety which is why there isn't a fixer for
>that.  Also, a fixer would have been really, really ugly (munge every
>class definition in this module because there is a top level
>assignment).

And almost certainly unnecessary.  IME, those are all to easily make bare
class definitions new style in Python 2.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/29b5ff24/attachment-0007.pgp>

From barry at python.org  Tue Jan 12 13:16:09 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Jan 2010 07:16:09 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
Message-ID: <20100112071609.1dcfffa6@freewill>

On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:

>Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
>That should work in Python 2.X and 3.X.

Ugly, but good call! :)

>I've got argparse running on Python 2.3-3.1, and the changes were
>pretty easy. You can see them all in the revision here:
>
>    http://code.google.com/p/argparse/source/detail?r=12
>
>I have aspirations of putting all of the tricks I learned up up on the
>Wiki somewhere, but I just haven't had the time.

The more resources we can provide people, both in code and in documentation,
the better.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/a2ba5b3e/attachment-0007.pgp>

From fuzzyman at voidspace.org.uk  Tue Jan 12 13:29:12 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 12:29:12 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112071609.1dcfffa6@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com>
	<20100112071609.1dcfffa6@freewill>
Message-ID: <4B4C6B18.7050008@voidspace.org.uk>

On 12/01/2010 12:16, Barry Warsaw wrote:
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>    
>> Actually there's a solution to this one too:
>>
>>     FooBase = Meta('FooBase', (), {})
>>     class Foo(FooBase):
>>         ...
>>
>> That should work in Python 2.X and 3.X.
>>      
> Ugly, but good call! :)
>
>    

There are all sorts of tricks. For example you can do exception handling 
that works with pre-2.6 syntax and 3.0 with a bare except and using 
sys.exc_info. It is horrible, but acceptable for short pieces of code (I 
have a couple of small modules that do this).

I haven't yet tried converting larger code-bases to Python 3, but I 
think the workflow advocated by Martin is greatly preferable to the 
hacks and tricks needed to make the same codebase run under 2 & 3.

Michael

>> I've got argparse running on Python 2.3-3.1, and the changes were
>> pretty easy. You can see them all in the revision here:
>>
>>     http://code.google.com/p/argparse/source/detail?r=12
>>
>> I have aspirations of putting all of the tricks I learned up up on the
>> Wiki somewhere, but I just haven't had the time.
>>      
> The more resources we can provide people, both in code and in documentation,
> the better.
>
> Thanks!
> -Barry
>    
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/7acd54f3/attachment-0007.htm>

From mal at egenix.com  Tue Jan 12 14:09:50 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 12 Jan 2010 14:09:50 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <4B4C749E.4040009@egenix.com>

Brett Cannon wrote:
> Michael has given me the hg transition/stdlib time slot at the language
> summit this year. In regards to that I plan to lead a discussion on:
> 
> * where we are at w/ the Hg transition (Dirkjan should be there and I did a
> blog post on this topic recently:
> http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> * argparse (PEP 389)
> * brief mention on still wanting to break out the stdlib from CPython
> * an official policy on extension modules? (i.e. must have a pure Python
> implementation which can mean a ctypes implementation unless given an
> explicit waiver)
> 
> That's everything from a stdlib perspective. I have half-baked ideas, but
> nothing concrete enough to bring up unless people really want to discuss
> stuff like how to potentially standardize module deprecation warnings, etc.
> 
> In terms of me personally, I do plan to bring up at some point during the
> summit these points which don't squarely fit during my time slot:
> 
> * an official unofficial policy on how new proposed features should come to
> us (i.e. working code to python-ideas with a shell of a PEP that includes
> abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> * any changes needed to the issue tracker to help with the workflow? (stage
> field seems like a failed experiment and we now have several effective
> triage people who can help w/ guiding changes)
> 
> If there is something missing from the stdlib discussion that you think
> should be brought up at the summit let me know. And if there is something
> here you want to discuss before the summit let me know and I can start a
> separate thread on it.

Could you please put these things and the results up on the Python
wiki ?!

We're going to have a language summit at EuroPython this year
as well and may want to continue/extend the discussion based
on what you're doing at PyCon.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From brian.curtin at gmail.com  Tue Jan 12 15:33:06 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 12 Jan 2010 08:33:06 -0600
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <hihida$unc$1@ger.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org>
Message-ID: <cf9f31f21001120633n6460b55as6064228cce41c949@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:20, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Mon, 11 Jan 2010 19:57:46 -0600, Brian Curtin a ?crit :
> >
> > For example, there are currently over
> > 1500 open issues with no stage set, some of which seemingly haven't been
> > read by anyone at all.
>
> I think most issues /have/ been read. It's just that for many of them,
> nobody is interested enough in or feels competent enough for fixing them.
>
> > Would a properly set stage field save issues from
> > falling into a black hole?
>
> I don't think this has anything to do with properly setting the stage
> field. We just have limited time and manpower. Perhaps one of our goals
> should be to reach out more to potential contributors.


Agreed, I didn't mean to place blame on the stage field, I just ran with how
I view that field since it was mentioned. When I'm thinking in "code-writing
developer mode" (tm), I'm more likely to go to issues which have a stage so
I know what I'm going into -- what needs to be worked on. When I'm in
project cleanup mode, I go by stageless issues -- what is necessary for this
to begin or end work. Maybe others work differently...software projects take
all kinds :-)

Anyways, sorry for the off-topic. If this is a summit worthy discussion,
cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/cf466340/attachment-0007.htm>

From rdmurray at bitdance.com  Tue Jan 12 17:12:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 11:12:28 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C66A6.2040601@gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<hihida$unc$1@ger.gmane.org> <4B4C66A6.2040601@gmail.com>
Message-ID: <20100112161228.300EA1D688D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 22:10:14 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> There are actually a whole host of reasons issues can stagnate:
> - a feature request may seem reasonable (hence it doesn't get rejected
> outright), but the right API may not be clear (hence it doesn't get
> implemented in the near term)
> - a patch may be reviewed and found to have significant defects (or
> simply be overreaching the stated goal) but the original patch poster
> loses interest after meeting resistance in their ambition to fix
> something that is "obviously" broken
> - a problem may simply be hard to fix in a backwards compatible way (or
> even at all!)

I would add:

- a patch is reviewed but the reviewer requests a second opinion before
  committing, which never arrives, and the original reviewer forgets
  about the issue.

I have occasionally observed apparently moribund issues spring to life
and get resolved when a triage person does something as simple as updating
the releases to which an issue applies.

> Ultimately it boils down to Antoine's two categories (lack of interest
> and lack of relevant expertise to make a final decision) but there are a
> lot of different ways to get into those two buckets.

There's a third bucket here: lack of time.  Sometimes there is interest
and expertise, but no time.  Worse, sometimes we end up waiting on the
person with the expertise and interest but no time, when someone else
with somewhat less expertise but more time should just go ahead and do
the commit.  (*That* one should be fixable.)

> Because we aren't ruthless about pruning those kinds of issues out of
> the tracker they're the ones that are going to accumulate over time.

Another other category that hangs around in the tracker is items for
which there simply isn't a consensus.  I suppose those could be brought
to python-dev for a final decision if someone wanted to tackle that task.

And I don't think it is a lack of ruthlessness.  I think it is the current
community consensus that we leave these issues open in the tracker in
case someone does come along and want to deal with them. (*)

> I'd actually be interested to know what the issue stats are like when
> RFEs are excluded and when the search is limited to the items flagged as
> 'easy'. If easy bug fix issues are taking a long time to get closed than
> that would bother me a lot more than RFEs that sit on the tracker for
> years.

I'd also be interested to know what the average and median lifetime
of closed bug reports is.  Not the metrics for open issues, but the
metrics for closed ones.  My theory is that we close a lot of bugs fairly
promptly, but the ones in the above categories make the average age of
*open* issues high.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls

(*) For example, I'm currently going through all the open issues
relating to the email package, and some of them are pretty darn
old, yet still have useful content.


From brett at python.org  Tue Jan 12 18:40:13 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:40:13 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <4B4C749E.4040009@egenix.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<4B4C749E.4040009@egenix.com>
Message-ID: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>

On Tue, Jan 12, 2010 at 05:09, M.-A. Lemburg <mal at egenix.com> wrote:

> Brett Cannon wrote:
> > Michael has given me the hg transition/stdlib time slot at the language
> > summit this year. In regards to that I plan to lead a discussion on:
> >
> > * where we are at w/ the Hg transition (Dirkjan should be there and I did
> a
> > blog post on this topic recently:
> > http://sayspy.blogspot.com/2010/01/where-hg-transition-stands.html)
> > * argparse (PEP 389)
> > * brief mention on still wanting to break out the stdlib from CPython
> > * an official policy on extension modules? (i.e. must have a pure Python
> > implementation which can mean a ctypes implementation unless given an
> > explicit waiver)
> >
> > That's everything from a stdlib perspective. I have half-baked ideas, but
> > nothing concrete enough to bring up unless people really want to discuss
> > stuff like how to potentially standardize module deprecation warnings,
> etc.
> >
> > In terms of me personally, I do plan to bring up at some point during the
> > summit these points which don't squarely fit during my time slot:
> >
> > * an official unofficial policy on how new proposed features should come
> to
> > us (i.e. working code to python-ideas with a shell of a PEP that includes
> > abstract and motivation -> hashed out PEP to python-dev -> pronouncement)
> > * any changes needed to the issue tracker to help with the workflow?
> (stage
> > field seems like a failed experiment and we now have several effective
> > triage people who can help w/ guiding changes)
> >
> > If there is something missing from the stdlib discussion that you think
> > should be brought up at the summit let me know. And if there is something
> > here you want to discuss before the summit let me know and I can start a
> > separate thread on it.
>
> Could you please put these things and the results up on the Python
> wiki ?!
>
> We're going to have a language summit at EuroPython this year
> as well and may want to continue/extend the discussion based
> on what you're doing at PyCon.
>
>
I expect there will be at least summary emails on what gets discussed. There
is also a chance that it will be videotaped.

-Brett


> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 12 2010)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/04964610/attachment-0007.htm>

From brett at python.org  Tue Jan 12 18:47:50 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 09:47:50 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>

On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:

> On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
>
>>  * any changes needed to the issue tracker to help with the workflow?
>> (stage field seems like a failed experiment and we now have several
>> effective triage people who can help w/ guiding changes)
>>
>> -Brett
>>
> I think it would be interesting to see how people are using the tracker, or
> how they want to be using it. For example, there are currently over 1500
> open issues with no stage set, some of which seemingly haven't been read by
> anyone at all. Would a properly set stage field save issues from falling
> into a black hole?
>
>
"Using" is the key word there. I know this thread went on about how bugs
tend to end up being left open, but what I was proposing to talk about was
whether there is any shift desired by the seasoned tracker users to help in
their work. For instance, the Stage field was meant to help, but I don't
think it is really getting used much, which makes it at best just a UI junk
and at worst something to confuse new users. So I just wanted to discuss
things as a group to see if we could streamline some things, add others,
etc.

At worst I will try to grab people like Mark D., R. David, Antoine, Georg,
etc. at the summit (not sure exactly which the heavy bug closers will be
there), take them aside, and flat-out ask what they need to make their jobs
easier.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/3f808530/attachment-0007.htm>

From brett at python.org  Tue Jan 12 19:29:06 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:29:06 -0800
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C6B18.7050008@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> 
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com> 
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org> 
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill> 
	<d11dcfba1001112157r5bbfbb7as8df6f802ba9cd167@mail.gmail.com> 
	<20100112071609.1dcfffa6@freewill> <4B4C6B18.7050008@voidspace.org.uk>
Message-ID: <bbaeab101001121029v28a863ccg8213ed494f15160c@mail.gmail.com>

On Tue, Jan 12, 2010 at 04:29, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>  On 12/01/2010 12:16, Barry Warsaw wrote:
>
> On Jan 11, 2010, at 09:57 PM, Steven Bethard wrote:
>
>
>
>  Actually there's a solution to this one too:
>
>    FooBase = Meta('FooBase', (), {})
>    class Foo(FooBase):
>        ...
>
> That should work in Python 2.X and 3.X.
>
>
>  Ugly, but good call! :)
>
>
>
>
> There are all sorts of tricks. For example you can do exception handling
> that works with pre-2.6 syntax and 3.0 with a bare except and using
> sys.exc_info. It is horrible, but acceptable for short pieces of code (I
> have a couple of small modules that do this).
>
> I haven't yet tried converting larger code-bases to Python 3, but I think
> the workflow advocated by Martin is greatly preferable to the hacks and
> tricks needed to make the same codebase run under 2 & 3.
>
>
In other words we need to pull together a HOWTO for Python source like the
one for extension modules that Benjamin wrote and make it rather prominently
linked from the Python 3 documentation index page.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/86cc3a21/attachment-0007.htm>

From rdmurray at bitdance.com  Tue Jan 12 19:31:23 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 12 Jan 2010 13:31:23 -0500
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com>
Message-ID: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>

On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> >>  * any changes needed to the issue tracker to help with the workflow?
> >> (stage field seems like a failed experiment and we now have several
> >> effective triage people who can help w/ guiding changes)
> >>
> > I think it would be interesting to see how people are using the tracker, or
> > how they want to be using it. For example, there are currently over 1500
> > open issues with no stage set, some of which seemingly haven't been read by
> > anyone at all. Would a properly set stage field save issues from falling
> > into a black hole?
> >
> "Using" is the key word there. I know this thread went on about how bugs
> tend to end up being left open, but what I was proposing to talk about was
> whether there is any shift desired by the seasoned tracker users to help in
> their work. For instance, the Stage field was meant to help, but I don't
> think it is really getting used much, which makes it at best just a UI junk
> and at worst something to confuse new users. So I just wanted to discuss
> things as a group to see if we could streamline some things, add others,
> etc.

Well, I for one find the stage field useful.  It reminds me at a glance
where the issue is at in the workflow (and 'not set' is a valid place
in the workflow that an issue sometimes stays in for a while for reason
other than not having been triaged yet).  I suspect that if we discussed
it as a group (we who make the most changes on the tracker) we could
improve it, and the interface in general.

By the way, you could talk to people who aren't going to be at the
summit on #python-dev; I think all the currently tracker-active people
hang out there on a regular basis.

I'll have to give some thought to what changes/improvements might be
most useful, now that I've been doing this for a while.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From brett at python.org  Tue Jan 12 19:34:02 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 10:34:02 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com> 
	<bbaeab101001120947j3f9ab23dj531b0b35399a3188@mail.gmail.com> 
	<20100112183123.71F4D1FBE4D@kimball.webabinitio.net>
Message-ID: <bbaeab101001121034m6de86385u4e0e72301e404a59@mail.gmail.com>

On Tue, Jan 12, 2010 at 10:31, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 12 Jan 2010 09:47:50 -0800, Brett Cannon wrote:
> > On Mon, Jan 11, 2010 at 17:57, Brian Curtin <brian.curtin at gmail.com>
> wrote:
> > > On Sun, Jan 10, 2010 at 14:25, Brett Cannon <brett at python.org> wrote:
> > >>  * any changes needed to the issue tracker to help with the workflow?
> > >> (stage field seems like a failed experiment and we now have several
> > >> effective triage people who can help w/ guiding changes)
> > >>
> > > I think it would be interesting to see how people are using the
> tracker, or
> > > how they want to be using it. For example, there are currently over
> 1500
> > > open issues with no stage set, some of which seemingly haven't been
> read by
> > > anyone at all. Would a properly set stage field save issues from
> falling
> > > into a black hole?
> > >
> > "Using" is the key word there. I know this thread went on about how bugs
> > tend to end up being left open, but what I was proposing to talk about
> was
> > whether there is any shift desired by the seasoned tracker users to help
> in
> > their work. For instance, the Stage field was meant to help, but I don't
> > think it is really getting used much, which makes it at best just a UI
> junk
> > and at worst something to confuse new users. So I just wanted to discuss
> > things as a group to see if we could streamline some things, add others,
> > etc.
>
> Well, I for one find the stage field useful.  It reminds me at a glance
> where the issue is at in the workflow (and 'not set' is a valid place
> in the workflow that an issue sometimes stays in for a while for reason
> other than not having been triaged yet).  I suspect that if we discussed
> it as a group (we who make the most changes on the tracker) we could
> improve it, and the interface in general.
>
> By the way, you could talk to people who aren't going to be at the
> summit on #python-dev; I think all the currently tracker-active people
> hang out there on a regular basis.
>
>
Sounds reasonable.


> I'll have to give some thought to what changes/improvements might be
> most useful, now that I've been doing this for a while.


How about everyone who is active on bugs.python.org give it some thought and
we can try to have a discussion at some point in the relatively near future.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/26a09128/attachment-0007.htm>

From ncoghlan at gmail.com  Tue Jan 12 21:58:49 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 06:58:49 +1000
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<4B4C749E.4040009@egenix.com>
	<bbaeab101001120940t5bff1499me3b673615bafeb04@mail.gmail.com>
Message-ID: <4B4CE289.6040709@gmail.com>

Brett Cannon wrote:
> I expect there will be at least summary emails on what gets discussed.
> There is also a chance that it will be videotaped.

The Wiki makes for a better summary archive though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Tue Jan 12 22:53:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:53:01 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100111191128.39020d89@freewill>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
Message-ID: <4B4CEF3D.8000107@v.loewis.de>

>> a) telling people that they have to move to 2.6 first actually
>>   hurts migration, instead of helping, because it implies to them
>>   that they have to drop old versions (e.g. 2.3.) - just because
>>   they had *always* dropped old versions before supporting new ones.
> 
> Is it just an implication, or is it reality?

That's only the implication. However, this was precisely the dialogue
when talking to Django. If you start with "start supporting 2.6", the
immediate response, without listening further, was, "ok, wait until we
drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).

Then explain it to the individual you are talking to, wait for the next
developer of the project step along, and see how he brings up the
very same line of thinking (supporting new versions == dropping support
for old versions).

I think only part of that comes from the maintenance burden. The other
part is that they *want* to drop support for old versions, so that they
can eventually start using new features (e.g. generator expressions).
So they welcome the requirement to support new versions as an excuse
to drop old ones ("it is obvious that you have to drop 2.3 to support
3.2"). However, their users then won't let them drop old versions.


> 
>> b) IMO, people also don't gain much by first migrating to 2.6.
>>   In principle, it gives them the opportunity to get py3k warnings.
>>   However, I haven't heard a single positive report where these
>>   warnings have actually helped people in porting. Yours is the
>>   first report saying that you followed the official guideline,
>>   but you didn't state whether doing so actually helped (or whether
>>   you just ported to 2.6 because the guideline told you to).
> 
> Python 2.6 has other useful features, which I want to take advantage of

I think you are a minority with that, being able to actually use the 2.6
features already. Many projects can't, as they have to support at least
2.4 still (so the with statement is right out).

>> Inherently, 2.8 can't improve on that.
> 
> I'm not so sure.  Yes, as a package maintainer there are older versions to
> think about, but time moves on for everyone (hopefully :).  By the time 2.8 is
> released, what will be the minimum version of Python provided by most OS
> vendors (where the majority of Python users probably get their 'python')?

"Current" Linux distributions will have 2.6 then. "Old" installations
will have 2.4.

> I
> guess some people will have to support everything from Python 2.3 to 2.8 but
> you're talking supporting something like a spread of 7 years of Python
> versions.  What other platform do you support for 7 years?

I think 2.3 will really be gone by the time 2.8 might get released. Even
with 2.7, you'd end up with a span of seven years, though.

Python had been supporting Windows 95 for more than 7 years (I think
rather 9 or 10), likewise Windows 3.1 before that. Python 2.7 will
likely still support Windows 2000, which then will be ten years old.

Solaris support will probably go back to Solaris 2.6, which will be
13 years old when Python 2.7 gets released.

It's only the Linux (and OS X) releases that move so quickly.

Regards,
Martin


From martin at v.loewis.de  Tue Jan 12 22:56:55 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 22:56:55 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
Message-ID: <4B4CF027.4080007@v.loewis.de>

> Maybe not, but the Distribute feature is there because IMO the
> distutils feature by itself isn't particularily useful. You need to
> write your own distutils extensions, in practice, and they are not
> trivial.

I wouldn't say that. My Django port works with bare distutils (as
does Django itself), and it works fine.

That distribute had to redo it is only because setuptools *replaces*
the build_py command, as does the 2to3 support in distutils. So only
if you have a different build_py already, you can't use what is in
distutils.

Regards,
Martin


From fuzzyman at voidspace.org.uk  Tue Jan 12 23:02:34 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:02:34 +0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CEF3D.8000107@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>
	<4B4CEF3D.8000107@v.loewis.de>
Message-ID: <4B4CF17A.1000503@voidspace.org.uk>

On 12/01/2010 21:53, "Martin v. L?wis" wrote:
>>> a) telling people that they have to move to 2.6 first actually
>>>    hurts migration, instead of helping, because it implies to them
>>>    that they have to drop old versions (e.g. 2.3.) - just because
>>>    they had *always* dropped old versions before supporting new ones.
>>>        
>> Is it just an implication, or is it reality?
>>      
> That's only the implication. However, this was precisely the dialogue
> when talking to Django. If you start with "start supporting 2.6", the
> immediate response, without listening further, was, "ok, wait until we
> drop 2.3, which will be in Spring 2009" (it has happened by now, IIUC).
>
> Then explain it to the individual you are talking to, wait for the next
> developer of the project step along, and see how he brings up the
> very same line of thinking (supporting new versions == dropping support
> for old versions).
>
> I think only part of that comes from the maintenance burden. The other
> part is that they *want* to drop support for old versions, so that they
> can eventually start using new features (e.g. generator expressions).
> So they welcome the requirement to support new versions as an excuse
> to drop old ones ("it is obvious that you have to drop 2.3 to support
> 3.2"). However, their users then won't let them drop old versions.
>
>
>    

I agree with Martin that the *perception* is that to use Python 2.6 to 
help you port to Python 3 you have to be willing to drop support for 
earlier versions of Python.

>>      
>>> b) IMO, people also don't gain much by first migrating to 2.6.
>>>    In principle, it gives them the opportunity to get py3k warnings.
>>>    However, I haven't heard a single positive report where these
>>>    warnings have actually helped people in porting. Yours is the
>>>    first report saying that you followed the official guideline,
>>>    but you didn't state whether doing so actually helped (or whether
>>>    you just ported to 2.6 because the guideline told you to).
>>>        
>> Python 2.6 has other useful features, which I want to take advantage of
>>      
> I think you are a minority with that, being able to actually use the 2.6
> features already. Many projects can't, as they have to support at least
> 2.4 still (so the with statement is right out).
>
>    
Well, the IronPython community has almost completely moved over to 
IronPython 2.6. :-)

We tend to ship our Python runtime with our applications though. As it 
happens I'm now working with CPython on the server (Python 2.5) and 
IronPython in the browser where we are using 2.6. The new property 
feature is nice, as is having with without a __future__ import.

All the best,


Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From regebro at gmail.com  Tue Jan 12 23:03:41 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 12 Jan 2010 23:03:41 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF027.4080007@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de> <20100111191128.39020d89@freewill>
	<319e029f1001112249u6104702fhf96457bddb44254a@mail.gmail.com>
	<4B4CF027.4080007@v.loewis.de>
Message-ID: <319e029f1001121403x14ef7ec8x729fb80fa67feaef@mail.gmail.com>

On Tue, Jan 12, 2010 at 22:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Maybe not, but the Distribute feature is there because IMO the
>> distutils feature by itself isn't particularily useful. You need to
>> write your own distutils extensions, in practice, and they are not
>> trivial.
>
> I wouldn't say that. My Django port works with bare distutils (as
> does Django itself), and it works fine.
>
> That distribute had to redo it is only because setuptools *replaces*
> the build_py command, as does the 2to3 support in distutils. So only
> if you have a different build_py already, you can't use what is in
> distutils.

Yeah, you are right, I misremembered. The actual additional feature is
the support for the test command. Testing under Python 2 and 3 without
it is annoying.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From martin at v.loewis.de  Tue Jan 12 23:04:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:04:37 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112000726.GO32351@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
Message-ID: <4B4CF1F5.4050403@v.loewis.de>

> [...]
>> I've done a fair bit of 3.x porting, and I'm firmly convinced that
>> 2.x can do nothing:
> [...]
>> Inherently, 2.8 can't improve on that.
> 
> I agree that there are limitations like the ones you've listed, but I
> disagree with your conclusion.  Maybe you assume that it's just as hard
> to move to 2.8 (using the py3k backports not available in say 2.5) as it
> is to 3.x?

Not at all, no. I'd rather expect that code that runs on 2.7 will run
on 2.8 unmodified.

> But a hypothetical 2.8 would also give people a way to move closer to
> py3k without giving up on using all their 2.x-only dependencies. 

How so? If they use anything that is new in 2.8, they *will* need to
drop support for anything before it, no???

> I think it's much more likely that libraries like Twisted can support 2.8
> in the near future than 3.x.

Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
that help Twisted in moving to 3.2?

> Then, when all of your dependencies (or viable alternatives to those
> dependencies) are available for 3.x, you'll have an easier transition if
> you can start from a 2.x with fewer differences in features.

But you won't *have* fewer differences. Just because your code runs
on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
for that). This doesn't get you any closer - you can't use any of
the 2.8 features as long as you have to support older versions of
Python.

> Fundamentally the more 2.x can converge on 3.x, the easier it will be
> for users to make the leap, because it will be a smaller leap.

No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
likely won't.

> The
> longer the 2.x series lives, the more these newer 2.x versions like 2.7
> and maybe even 2.8 will be available on common platforms for people to
> depend upon as minimum versions, which means that as time goes by they
> can depend on a version that's closer to 3.x.

No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
released in 2010. Then, in 2015, it may be that everybody has migrated
to 2.7, which then is a common platform.

If you release 2.8 in 2012, then, in 2015, people will be split between
2.7 and 2.8, and so there won't be a common platform before 2017.

So stopping 2.x development *earlier* will also give us a common
platform earlier.

Regareds,
Martin


From martin at v.loewis.de  Tue Jan 12 23:07:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 12 Jan 2010 23:07:02 +0100
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
	<cf9f31f21001111757rae32637id6f4df97ab4c8dc5@mail.gmail.com>
Message-ID: <4B4CF286.5040002@v.loewis.de>

> I think it would be interesting to see how people are using the tracker,
> or how they want to be using it. For example, there are currently over
> 1500 open issues with no stage set, some of which seemingly haven't been
> read by anyone at all. Would a properly set stage field save issues from
> falling into a black hole?

I personally don't think this would be the case.

What would help is people who actually *work* on the issues, rather than
merely reading them. However, this being open source, there is no way to
force it (unless you create an incentive, such as the five-for-one
deal).

Regards,
Martin


From python at mrabarnett.plus.com  Tue Jan 12 23:10:28 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 12 Jan 2010 22:10:28 +0000
Subject: [Python-Dev] regex module
Message-ID: <4B4CF354.2050603@mrabarnett.plus.com>

Hi all,

I'm back on the regex module after doing other things and I'd like your
opinion on a number of matters:

Firstly, the current re module has a bug whereby it doesn't split on
zero-width matches. The BDFL has said that this behaviour should be
retained by default in case any existing software depends on it. My
question is: should my regex module still do this for Python 3?
Speaking personally, I'd like it to behave correctly, and Python 3 is
the version where backwards-compatibility is allowed to be broken.

Secondly, Python 2 is reaching the end of the line and Python 3 is the
future. Should I still release a version that works with Python 2? I'm
thinking that it could be confusing if new regex module did zero-width
splits correctly in Python 3 but not in Python 2. And also, should I
release it only for Python 3 as a 'carrot'?

Finally, the module allows some extra backslash escapes, eg \g<name>, in
the pattern. Should it treat ill-formed escapes, eg \g, as it would have
treated them in the re module?

Thanks


From michael at voidspace.org.uk  Tue Jan 12 23:46:49 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 22:46:49 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
Message-ID: <4B4CFBD9.1090009@voidspace.org.uk>

I presume the email below is about the Windows binary. Does the AMD64 
release work on intel 64bit and can we make the wording clearer on the 
download page?

The current description is " Windows AMD64 binary".

All the best,

Michael

-------- Original Message --------
Subject: 	Download Page - AMD64
Date: 	Fri, 11 Dec 2009 04:03:16 -0600
From: 	Thomas Brownback <thomas.brownback at gmail.com>
To: 	webmaster at python.org



Should I use the AMD64 version of Python on an Intel 64 chip? I know 
those 64-bit implementations are very similar, but are they similar 
enough that your AMD64 will work on Intel?

If so, perhaps a note should be placed on that page to help avoid 
confusion. Explicit being better than implicit and all.

Thanks for your time,
Thomas

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/80d6b539/attachment-0007.htm>

From andrew at bemusement.org  Tue Jan 12 23:49:56 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Wed, 13 Jan 2010 09:49:56 +1100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100112224956.GC28576@steerpike.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > But a hypothetical 2.8 would also give people a way to move closer to
> > py3k without giving up on using all their 2.x-only dependencies. 
> 
> How so? If they use anything that is new in 2.8, they *will* need to
> drop support for anything before it, no???
> 
> > I think it's much more likely that libraries like Twisted can support 2.8
> > in the near future than 3.x.
> 
> Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
> that help Twisted in moving to 3.2?

I'm not talking about Twisted moving to 3.x (FWIW, I think the only
movement there so far is some patches for some -3 warnings).  The
situation I'm describing is a project X that:

  (a) has 2.x-only dependencies, and
  (b) would like to be as close as possible to 3.x (because they like
      the new features and/or want to be as ready as possible to jump
      when (a) is fixed).

So just because project X depends on e.g. Twisted, and that Twisted in
turn still supports 2.4, doesn't mean that X cannot move to 2.8, and
doesn't mean it would get no benefit from doing so.

[...]
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.

But this is my point.  I think they would as an intermediate step to
jumping to 3.x (which also requires dropping 2.5, after all!), if for
some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

-Andrew.



From tjreedy at udel.edu  Tue Jan 12 23:51:31 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 17:51:31 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <hiiudf$2uv$1@ger.gmane.org>

On 1/12/2010 5:04 PM, "Martin v. L?wis" wrote:

> But you won't *have* fewer differences. Just because your code runs
> on 2.8 doesn't mean it will stop running on 2.3 (if you have a need
> for that). This doesn't get you any closer - you can't use any of
> the 2.8 features as long as you have to support older versions of
> Python.
>
>> Fundamentally the more 2.x can converge on 3.x, the easier it will be
>> for users to make the leap, because it will be a smaller leap.
>
> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
> likely won't.
>
>> The
>> longer the 2.x series lives, the more these newer 2.x versions like 2.7
>> and maybe even 2.8 will be available on common platforms for people to
>> depend upon as minimum versions, which means that as time goes by they
>> can depend on a version that's closer to 3.x.
>
> No, that's incorrect. Suppose 2.7 is the last 2.x release, to be
> released in 2010. Then, in 2015, it may be that everybody has migrated
> to 2.7, which then is a common platform.
>
> If you release 2.8 in 2012, then, in 2015, people will be split between
> 2.7 and 2.8, and so there won't be a common platform before 2017.

Just like people today may need to work with both 2.5 and 2.6, or 
privately backport 2.6 bugfixes to 2.5.

> So stopping 2.x development *earlier* will also give us a common
> platform earlier.

With years of bug fixes and hence high quality.




From tjreedy at udel.edu  Wed Jan 13 00:05:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 Jan 2010 18:05:12 -0500
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <hiiv75$5md$1@ger.gmane.org>

On 1/12/2010 5:10 PM, MRAB wrote:
> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.

Are you writing a new module with a new name? If so, do you expect it to 
replace or augment re? (This is the same question as for optparse vs. 
argparse, which I understand to not yet be decided.)
>
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?

2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
2.7 stdlib is already out. A new engine should get some community 
testing before going in the stdlib. Even 3.2 beta is not that far off 
(8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?

> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?

What does re do with analogous cases?

Terry Jan Reedy



From david.lyon at preisshare.net  Wed Jan 13 00:20:54 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 13 Jan 2010 10:20:54 +1100 (EST)
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4C4589.70906@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org>
	<bbaeab101001101209w2e6d8c60k557f1619fe451afc@mail.gmail.com>
	<20100111014457.GA5289@arctrix.com>
	<bbaeab101001101946r1e1094b8o793dc62dfd8b5fe0@mail.gmail.com>
	<20100111040507.GA7200@arctrix.com>
	<bbaeab101001102106k253f43bbgbe7f058c536c279d@mail.gmail.com>
	<20100111052713.GA8211@arctrix.com> <4B4ADED6.5080207@v.loewis.de>
	<1179.218.214.45.58.1263252162.squirrel@syd-srv02.ezyreg.com>
	<4B4C4589.70906@gmail.com>
Message-ID: <1333.218.214.45.58.1263338454.squirrel@syd-srv02.ezyreg.com>

> Nick wrote:
>>> This has nothing to do with pushing 3.x, but all with managing
>>> available manpower and still providing quality software.
>>
>> Python 3.x needs more carrots.
>
> As Guido has said a few times, the gains are far greater for *new*
> Python developers than they are for existing ones.

Well both you and Guido are most likely 100% correct. :-)

> They don't have as rich a 3rd party ecosystem
> on Python 3 as they would on Python 2.x at this point in time, but
> unlike existing developers they also have the luxury of cherry-picking
> just those packages that already have Python 3 support.

Most likely. I wouldn't want to say anything to discourage
people from going to python 3. In other languages, I have
much experience of making the jump to 'a whole new world'.

It's unsettling at first, but after that, you suddenly
find the new model has a better turbo than the last and
you're at the next corner faster than you can think. So
it's all good.

But I still maintain Python 3.0 needs more carrots. For example,
if mercurial or any other cool lib gets added to python 3 (and
I can name a few that should go in python 3) then they should
be added to python 3 and not to python 2.x

They would serve as good carrots.

Make fresh the python 3 stdlib and preserve the python 2.x stdlib.

I really think we are somewhat short on resources to do
what Guido has asked about bringing python up to CPAN level
with respect to packages.

We're starting a long way back with horses and swords and
trying to catch a well fed and greased modern machine..

I'm sure we can get a modern package testbot operational
for python.

But I wish those who were complaining about the packaging
problem so much could throw some money at the problem
instead of moaning about it. As is done on other open-source
projects. Some organisation would be beneficial.

Finding funds so that a small group of people could
work on the problem would be a great boost to forward
progress. I just think python packaging needs six
months of that to repair the years and years of neglect
and stagnation. Even if it is only beer and bus fare
money. It just needs a temporary shot of adrenalin in
the arm.. so to speak..

Regards

David











From martin at v.loewis.de  Wed Jan 13 00:28:14 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 00:28:14 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D058E.4030404@v.loewis.de>

> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?

"intel 64bit" is as clear as mud. It could mean the "Intel 64"
architecture, or it could mean the "IA-64" architecture, both
are 64-bit architectures with processors manufactured by Intel.
The former is officially called AMD64 (not by Intel, but
officially), and our AMD64 binaries run on such processors.
The latter is also called "Itanium", and we stopped producing
binaries for that architecture a while ago; the AMD64 binaries
will *not* run on it.

The wording could probably be changed to match the reader's
expectation more, most likely, it would also get more incorrect
in the process. To make it correct and explicit, an entire
paragraph educating users about 64-bit architectures and that
there are many of them and that they are mutually incompatible
might be required.

Most likely, Thomas' processor implements the AMD64 architecture,
even though it is manufactured by Intel. A short sentence that
he would probably understand (given that he used "Intel 64", not
"64bit") is:

"""The binaries for AMD64 will also work on processors that implement
the Intel 64 architecture (formerly EM64T), i.e. the architecture that
Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
They will not work on Intel Itanium Processors (formerly IA-64)."""

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 00:30:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 00:30:27 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <20100112224956.GC28576@steerpike.home.puzzling.org>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>
	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100112000726.GO32351@steerpike.home.puzzling.org>	<4B4CF1F5.4050403@v.loewis.de>
	<20100112224956.GC28576@steerpike.home.puzzling.org>
Message-ID: <4B4D0613.1010503@v.loewis.de>

> I'm not talking about Twisted moving to 3.x (FWIW, I think the only
> movement there so far is some patches for some -3 warnings).  The
> situation I'm describing is a project X that:
> 
>   (a) has 2.x-only dependencies, and
>   (b) would like to be as close as possible to 3.x (because they like
>       the new features and/or want to be as ready as possible to jump
>       when (a) is fixed).

This assumes that jumping to 3.x is easier if you are closer to it.
Please trust me that this is not the case.

>> No, it won't. It might be if people move to 2.8 *and* drop 2.5, but they
>> likely won't.
> 
> But this is my point.  I think they would as an intermediate step to
> jumping to 3.x (which also requires dropping 2.5, after all!), if for
> some reason they cannot yet jump to 3.x, such as a 2.x-only dependency.

No, and no. No: it's not an intermediate step, and no, supporting 3.x
does *not* require dropping 2.5.

Regards,
Martin



From fuzzyman at voidspace.org.uk  Wed Jan 13 00:40:35 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:40:35 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D058E.4030404@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
Message-ID: <4B4D0873.5070006@voidspace.org.uk>

On 12/01/2010 23:28, "Martin v. L?wis" wrote:
> [snip...]
> """The binaries for AMD64 will also work on processors that implement
> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
> They will not work on Intel Itanium Processors (formerly IA-64)."""
>
>    

To those not intimately aware of the history and present of 64 bit 
architecture this sentence would be a great addition - thanks. I'm 
adding it now as a footnote.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From lists at cheimes.de  Wed Jan 13 00:41:04 2010
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 13 Jan 2010 00:41:04 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D0890.2030505@cheimes.de>

Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64 
> release work on intel 64bit and can we make the wording clearer on the 
> download page?
> 
> The current description is " Windows AMD64 binary".

The installer works on all AMD64 compatible Intel CPUs. *scnr*

As you most likely know all modern Intel 64bit CPUs are based on AMD's
x86-64 design. Only the Itanium family is based on the other Intel 64bit
design: IA-64. The name AMD64 was chosen because most Linux and BSD
distributions call it so. The name 'AMD64' has caused confusion in the
past because users thought that the installer works only on AMD CPUs.

How about:

 * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
X86-64 binary -- does not include source)

instead of:

 * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
not include source)

?

Christia


From fuzzyman at voidspace.org.uk  Wed Jan 13 00:55:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 12 Jan 2010 23:55:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0873.5070006@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D058E.4030404@v.loewis.de>
	<4B4D0873.5070006@voidspace.org.uk>
Message-ID: <4B4D0BD4.5050109@voidspace.org.uk>

On 12/01/2010 23:40, Michael Foord wrote:
> On 12/01/2010 23:28, "Martin v. L?wis" wrote:
>> [snip...]
>> """The binaries for AMD64 will also work on processors that implement
>> the Intel 64 architecture (formerly EM64T), i.e. the architecture that
>> Microsoft calls x64, and AMD called x86-64 before calling it AMD64.
>> They will not work on Intel Itanium Processors (formerly IA-64)."""
>>
>
> To those not intimately aware of the history and present of 64 bit 
> architecture this sentence would be a great addition - thanks. I'm 
> adding it now as a footnote.
>

I can add it now to the main download page and existing release pages - 
but are there changes needed to any scripts to change pages created for 
*new* releases?

All the best,

Michael Foord

> All the best,
>
> Michael Foord
>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>>
>
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From python at mrabarnett.plus.com  Wed Jan 13 01:22:01 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 00:22:01 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <hiiv75$5md$1@ger.gmane.org>
References: <4B4CF354.2050603@mrabarnett.plus.com> <hiiv75$5md$1@ger.gmane.org>
Message-ID: <4B4D1229.70708@mrabarnett.plus.com>

Terry Reedy wrote:
> On 1/12/2010 5:10 PM, MRAB wrote:
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
> 
> Are you writing a new module with a new name? If so, do you expect it to 
> replace or augment re? (This is the same question as for optparse vs. 
> argparse, which I understand to not yet be decided.)
 >
It's a module called 'regex'. It can be used in place of 're' by using
"import regex as re", except for differences such as "\g<name>" being a
legal group reference in pattern strings.
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
> 
> 2.7 is in alpha with no plans for 2.8, so unless you finish real soon, 
> 2.7 stdlib is already out. A new engine should get some community 
> testing before going in the stdlib. Even 3.2 beta is not that far off 
> (8-9 months?) Do *you* want to do the extra work for a 2.x release on PyPI?
> 
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
> 
> What does re do with analogous cases?
> 
The 're' module treats r"\g" as "g"; both 're' and 'regex' treat, say, 
r"\q" as "q". The closest analogue to what I'm asking about is that re
treats the ill-formed repeat r"x{1," as a literal, which sort of
suggests that r"\g" should be treated as "g", but r"\g<name>" is now a
group reference (re would treat that as "g<name>". Does that sound
reasonable?


From fuzzyman at voidspace.org.uk  Wed Jan 13 01:50:39 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 00:50:39 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D18DF.6070502@voidspace.org.uk>

On 12/01/2010 23:41, Christian Heimes wrote:
> Michael Foord wrote:
>    
>> I presume the email below is about the Windows binary. Does the AMD64
>> release work on intel 64bit and can we make the wording clearer on the
>> download page?
>>
>> The current description is " Windows AMD64 binary".
>>      
> The installer works on all AMD64 compatible Intel CPUs. *scnr*
>
> As you most likely know all modern Intel 64bit CPUs are based on AMD's
> x86-64 design. Only the Itanium family is based on the other Intel 64bit
> design: IA-64. The name AMD64 was chosen because most Linux and BSD
> distributions call it so. The name 'AMD64' has caused confusion in the
> past because users thought that the installer works only on AMD CPUs.
>
> How about:
>
>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
>
> instead of:
>
>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)
>    
Right - I've made that change for the Python 2.6, 2.7, 3.0 and 3.1 
download pages with a footnote. Prior to that we were offering ia64 
*and* amd64 releases so I've left those unchanged.

Thanks,

Michael

> ?
>
> Christia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From exarkun at twistedmatrix.com  Wed Jan 13 02:40:03 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 13 Jan 2010 01:40:03 -0000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF1F5.4050403@v.loewis.de>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<hid9s1$i05$1@ger.gmane.org> <4B4A373F.9050601@gmail.com>
	<4B4A5DCE.3070909@v.loewis.de>
	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>
	<4B4B9B55.1010709@v.loewis.de>
	<20100112000726.GO32351@steerpike.home.puzzling.org>
	<4B4CF1F5.4050403@v.loewis.de>
Message-ID: <20100113014003.1898.1305834328.divmod.xquotient.219@localhost.localdomain>

On 12 Jan, 10:04 pm, martin at v.loewis.de wrote:
>>[...]
>>>I've done a fair bit of 3.x porting, and I'm firmly convinced that
>>>2.x can do nothing:
>>[...]
>>>Inherently, 2.8 can't improve on that.
>>
>>I agree that there are limitations like the ones you've listed, but I
>>disagree with your conclusion.  Maybe you assume that it's just as 
>>hard
>>to move to 2.8 (using the py3k backports not available in say 2.5) as 
>>it
>>is to 3.x?
>
>Not at all, no. I'd rather expect that code that runs on 2.7 will run
>on 2.8 unmodified.
>>But a hypothetical 2.8 would also give people a way to move closer to
>>py3k without giving up on using all their 2.x-only dependencies.
>
>How so? If they use anything that is new in 2.8, they *will* need to
>drop support for anything before it, no???
>>I think it's much more likely that libraries like Twisted can support 
>>2.8
>>in the near future than 3.x.
>
>Most likely, Twisted "supports" 2.8 *today* (hopefully). But how does
>that help Twisted in moving to 3.2?

I'm not reading this thread carefully enough to make any arguments on 
either side, but I can contribute a fact.

Twisted very likely does not support 2.8 today.  I base this on the fact 
that Twisted does not support 2.7 today, and I expect 2.8 will be more 
like 2.7 than it will be like 2.3 - 2.6 (which Twisted does support).

When I say "support" here, I mean "all of the Twisted unit tests pass on 
it".

Jean-Paul


From tseaver at palladion.com  Wed Jan 13 03:57:46 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 12 Jan 2010 21:57:46 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
Message-ID: <4B4D36AA.7020607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Karen Tracey wrote:
> On Sat, Jan 9, 2010 at 12:29 PM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> On behalf of the Python development team, I'm gleeful to announce the
>> second
>> alpha release of Python 2.7.
>>
>>
> Well yay.  Django's test suite (1242 tests) runs with just one failure on
> the 2.7 alpha 2 level, and that looks to be likely due to the improved
> string/float rounding so not really a problem, just a difference.  That's
> down from 104 failures and 40 errors with 2.7 alpha 1.
> 
> Note on the website page http://www.python.org/download/releases/2.7/ the
> "Change log for this release" link is still pointing to the alpha 1
> changelog.

Just to add another "success" data point:  Zope2's trunk, as well as the
2.12 release, passes all tests (2535 on the trunk) and brings up the
appserver just fine under 2.7a2.

There is an obnoxious deprecation warning out of the distutils:

  DeprecationWarning: 'compiler' specifies the compiler type in
  build_ext. If you want to get the compiler object itself, use
  'compiler_obj'

which is likely a simple one-line fix, if I only knew what the hell it
was whining about. ;)  The warning is extra obnoxious because it doesn't
tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
argument).



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktNNqQACgkQ+gerLs4ltQ7d5gCcCGKVjyOlxnrAln0UnRibS7kd
lNIAoIs1RlSGMtJWaY11BqptfDmQvR87
=mIOO
-----END PGP SIGNATURE-----



From python at mrabarnett.plus.com  Wed Jan 13 04:09:34 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 Jan 2010 03:09:34 +0000
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <4B4D396E.4050500@mrabarnett.plus.com>

MRAB wrote:
> Hi all,
> 
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
> 
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
> 
> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
> 
> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
> 
I've just noticed something odd about the re module: the sub() method
doesn't take 'pos' or 'endpos' arguments. search() does; match() does;
findall() does(); finditer() does; but sub() doesn't. Maybe there has
never been a demand for it. (Nor split(), for that matter.)


From brett at python.org  Wed Jan 13 04:58:11 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 12 Jan 2010 19:58:11 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <4B4CF354.2050603@mrabarnett.plus.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
Message-ID: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>

On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:

> Hi all,
>
> I'm back on the regex module after doing other things and I'd like your
> opinion on a number of matters:
>
> Firstly, the current re module has a bug whereby it doesn't split on
> zero-width matches. The BDFL has said that this behaviour should be
> retained by default in case any existing software depends on it. My
> question is: should my regex module still do this for Python 3?
> Speaking personally, I'd like it to behave correctly, and Python 3 is
> the version where backwards-compatibility is allowed to be broken.
>
>
If it is a separate module under a different name it can do the proper
thing. People will just need to be aware of the difference when they import
the module.


> Secondly, Python 2 is reaching the end of the line and Python 3 is the
> future. Should I still release a version that works with Python 2? I'm
> thinking that it could be confusing if new regex module did zero-width
> splits correctly in Python 3 but not in Python 2. And also, should I
> release it only for Python 3 as a 'carrot'?
>
>
That's totally up to you. There is practically no chance of it getting into
the 2.x under the stdlib at this point since 2.7b1 is coming up and this
module has not been out in the wild for a year (to my knowledge).  If you
want to support 2.x that's fine and I am sure users would appreciate it, but
it isn't necessary to get into the Python 3 stdlib.


> Finally, the module allows some extra backslash escapes, eg \g<name>, in
> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
> treated them in the re module?
>
>
If you want to minimize the differences then it should probably match. As I
said, since it is a different name to import under it can deviate where
reasonable, just make sure to clearly document the deviations.

-Brett



> Thanks
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100112/f1f73d56/attachment-0007.htm>

From martin at v.loewis.de  Wed Jan 13 07:11:49 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 07:11:49 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D0890.2030505@cheimes.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <4B4D6425.3060307@v.loewis.de>

> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)
> 
> instead of:
> 
>  * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
> not include source)

-1. AMD doesn't want us to use the term x86-64 anymore, but wants us
to use AMD64 instead. I think we should comply - they invented the
architecture, so they have the right to give it a name. Neither
Microsoft nor Intel have such a right.

Regards,
Martin


From sridharr at activestate.com  Wed Jan 13 07:47:38 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 12 Jan 2010 22:47:38 -0800
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4CFBD9.1090009@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk>
Message-ID: <4B4D6C8A.7070307@activestate.com>

On 1/12/2010 2:46 PM, Michael Foord wrote:
> I presume the email below is about the Windows binary. Does the AMD64
> release work on intel 64bit and can we make the wording clearer on the
> download page?
>
> The current description is " Windows AMD64 binary".

FWIW, we simply use (64-bit, x64).

   Platform	          Download
   ==============================================
   Windows (x86)	          Windows Installer (MSI)
   Windows (64-bit, x64)	  Windows Installer (MSI)

https://www.activestate.com/activepython/downloads/

-srid


From solipsis at pitrou.net  Wed Jan 13 08:08:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 13 Jan 2010 07:08:55 +0000 (UTC)
Subject: [Python-Dev] Fwd: Download Page - AMD64
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
Message-ID: <loom.20100113T080717-468@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> How about:
> 
>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
> X86-64 binary -- does not include source)

+1. I don't care about trademarks or official names, we should call it whatever
is obvious for our users.
As for Itanium, it is practically dead, I think there is little chance of people
mixing it up with x86-64.

cheers

Antoine.




From michael at voidspace.org.uk  Wed Jan 13 10:32:00 2010
From: michael at voidspace.org.uk (Michael Foord)
Date: Wed, 13 Jan 2010 09:32:00 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <4B4D9310.6060907@voidspace.org.uk>

On 13/01/2010 06:11, "Martin v. L?wis" wrote:
>> How about:
>>
>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
>>
>> instead of:
>>
>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>> not include source)
>>      
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.
>    

I think we should use whatever is most informative and least confusing 
to our users - we owe our allegiance to them and not to a processor vendor.

Michael


> Regards,
> Martin
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ncoghlan at gmail.com  Wed Jan 13 11:30:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:30:50 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4CF17A.1000503@voidspace.org.uk>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk>
Message-ID: <4B4DA0DA.7070007@gmail.com>

Michael Foord wrote:
> I agree with Martin that the *perception* is that to use Python 2.6 to
> help you port to Python 3 you have to be willing to drop support for
> earlier versions of Python.

Note that increased 3.x compatibility in the most recent 2.x release
will always help in two scenarios:

1. New projects that want to use 2.x only libraries but want to be ready
for the Py3k transition in their own code (e.g. new 2.7 features like
set literals, dict and set comprehensions and the view* dict methods can
be translated far more cleanly by 2to3 than the closest comparable 2.6
code).

2. Similarly new projects that use a 3.x trunk can be more readily
translated to a 2.7 version with 3to2, whereas some constructs may be
difficult to recreate in earlier Python versions. I would expect this to
be significantly less common then the first scenario though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:38:31 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:38:31 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <loom.20100113T080717-468@post.gmane.org>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<loom.20100113T080717-468@post.gmane.org>
Message-ID: <4B4DA2A7.2080408@gmail.com>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> How about:
>>
>>  * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>> X86-64 binary -- does not include source)
> 
> +1. I don't care about trademarks or official names, we should call it whatever
> is obvious for our users.

Until this discussion, I didn't even know the architecture had any name
other than x86-64.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Wed Jan 13 11:39:40 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Jan 2010 20:39:40 +1000
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4D36AA.7020607@palladion.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com>
Message-ID: <4B4DA2EC.3050908@gmail.com>

Tres Seaver wrote:
> There is an obnoxious deprecation warning out of the distutils:
> 
>   DeprecationWarning: 'compiler' specifies the compiler type in
>   build_ext. If you want to get the compiler object itself, use
>   'compiler_obj'
> 
> which is likely a simple one-line fix, if I only knew what the hell it
> was whining about. ;)  The warning is extra obnoxious because it doesn't
> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
> argument).

Could you kick a tracker issues in Tarek's direction for that one?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From vinay_sajip at yahoo.co.uk  Wed Jan 13 13:24:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 13 Jan 2010 12:24:09 +0000 (UTC)
Subject: [Python-Dev] topics I plan to discuss at the language summit
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com>
Message-ID: <loom.20100113T131816-515@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> If there is something missing from the stdlib discussion that you think should
be brought up at the summit let me know. And if there is something here you want
to discuss before the summit let me know and I can start a separate thread on it.

I'm sorry I won't be able to come to the language summit, but I would like if
possible to expedite a pronouncement on PEP 391 (configuring logging using
dictionaries). I believe I addressed all the comments made on the discussion
threads mentioned in the PEP and so I'm not sure what more I need to do to get a
pronouncement. I guess the stdlib slot gives an opportunity for people to air
their views and so I'd be grateful if you added it to the agenda.

Thanks & regards,

Vinay Sajip



From murman at gmail.com  Wed Jan 13 14:35:51 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 07:35:51 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D6425.3060307@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
Message-ID: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>

On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
> to use AMD64 instead. I think we should comply - they invented the
> architecture, so they have the right to give it a name. Neither
> Microsoft nor Intel have such a right.

I see and agree with the motivation behind your point, but it's just
as reasonable to mimic the name Microsoft uses: the release is for
Windows rather than the processor. On MSDN Microsoft uses
parentheticals x86, ia64 and x64; this would suggest a name like
Python 2.6.4 installer for Windows (x64). Unfortunately this usage
doesn't seem to be reflected in consumer-oriented product pages, so
I'm uncertain how clear it is for those downloading Python.

-- 
Michael Urman


From ncoghlan at gmail.com  Wed Jan 13 15:04:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 00:04:28 +1000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
Message-ID: <4B4DD2EC.3030709@gmail.com>

Michael Urman wrote:
> On Wed, Jan 13, 2010 at 00:11, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
> 
> I see and agree with the motivation behind your point, but it's just
> as reasonable to mimic the name Microsoft uses: the release is for
> Windows rather than the processor. On MSDN Microsoft uses
> parentheticals x86, ia64 and x64; this would suggest a name like
> Python 2.6.4 installer for Windows (x64). Unfortunately this usage
> doesn't seem to be reflected in consumer-oriented product pages, so
> I'm uncertain how clear it is for those downloading Python.

As Windows doesn't run on non-x86 architectures, their naming is
generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Linux, on the other hand, can run on multiple 64 bit architectures, so
being more specific and using the official AMD64 nomenclature makes sense.

In this case, making it clear that the 64-bit Windows binaries work for
both Intel and AMD chips is more important than using only the official
architecture name.

Cheers,
Nick.

P.S. This discussion made me realise that out of habit I had dropped the
32-bit version of Python on my new 64-bit Windows box...

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From tseaver at palladion.com  Wed Jan 13 17:49:55 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 13 Jan 2010 11:49:55 -0500
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA2EC.3050908@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<af3536271001091048v22a79a01h105e1305530dacb@mail.gmail.com>
	<4B4D36AA.7020607@palladion.com> <4B4DA2EC.3050908@gmail.com>
Message-ID: <4B4DF9B3.4030403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> There is an obnoxious deprecation warning out of the distutils:
>>
>>   DeprecationWarning: 'compiler' specifies the compiler type in
>>   build_ext. If you want to get the compiler object itself, use
>>   'compiler_obj'
>>
>> which is likely a simple one-line fix, if I only knew what the hell it
>> was whining about. ;)  The warning is extra obnoxious because it doesn't
>> tell me what in *my* code triggers the warning (it (needs a 'stacklevel'
>> argument).
>
> Could you kick a tracker issues in Tarek's direction for that one?

Done:

  http://bugs.python.org/issue7694


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktN+bMACgkQ+gerLs4ltQ6s4gCdENhtVQWzoH449BCDz8SNx/4s
DEoAn19LMcMs3b6ctdNF8K2hYd6d+BSZ
=TWit
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Jan 13 18:24:42 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 13 Jan 2010 12:24:42 -0500
Subject: [Python-Dev] PYTHON3PATH
Message-ID: <20100113172442.4A7771FA679@kimball.webabinitio.net>

Please review issue 2375 [1], which is an enhancement request to add a
PYTHON3PATH environment variable.  Because we have elected to have both
a python and a python3 command, I think this is an issue worth thinking
about carefully to make sure we are serving the Python user community
and easing the transition to python3.  It could be that "use virtualenv"
is the best answer, but I feel we should think about it carefully to
make sure that is really true.

[1] http://bugs.python.org/issue2375

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From guido at python.org  Wed Jan 13 18:31:39 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:31:39 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>

Somehow the bug site doesn't load for me right now, but I'm -1 on
this. There are maybe a dozen PYTHON... variables -- we really
shouldn't try to add PYTHON3 variants for all of them.

Specifically for PYTHONPATH, I feel that its use is always a
short-time or localized hack, not something you set in your .profile
nd forget about it (forgetting about it inparticular often leads to
unnecessary debugging pain later). The better approaches are based on
site-packages, wrapper scripts, virtualenv, etc.

--Guido

On Wed, Jan 13, 2010 at 9:24 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable. ?Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3. ?It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375
>
> --
> R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com
> Business Process Automation - Network/Server Management - Routers/Firewalls
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)


From dirkjan at ochtman.nl  Wed Jan 13 18:38:26 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 13 Jan 2010 18:38:26 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>
Message-ID: <ea2499da1001130938v68827914yc41476c75b5f3c62@mail.gmail.com>

On Sat, Jan 9, 2010 at 18:29, Benjamin Peterson <benjamin at python.org> wrote:
> Please consider trying Python 2.7 with your code and reporting any bugs you may

I ran the Mercurial test suite on current trunk, and it worked
alright. There was a small issue with the fact that mimetypes got
fooled by the lack of ImportError from _winreg (which I solved by
blacklisting _winreg in demandimport), and one test fails likely
because of a change in MIME handling. Will look into that. So the
state of trunk looks rather solid from where I sit.

Cheers,

Dirkjan


From ralf at brainbot.com  Wed Jan 13 18:40:04 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:40:04 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113172442.4A7771FA679@kimball.webabinitio.net> (R. David
	Murray's message of "Wed, 13 Jan 2010 12:24:42 -0500")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
Message-ID: <87r5ptdhu3.fsf@muni.brainbot.com>

"R. David Murray" <rdmurray at bitdance.com> writes:

> Please review issue 2375 [1], which is an enhancement request to add a
> PYTHON3PATH environment variable.  Because we have elected to have both
> a python and a python3 command, I think this is an issue worth thinking
> about carefully to make sure we are serving the Python user community
> and easing the transition to python3.  It could be that "use virtualenv"
> is the best answer, but I feel we should think about it carefully to
> make sure that is really true.
>
> [1] http://bugs.python.org/issue2375

The first thing I got while trying to run a python3 prompt few days ago,
was an error. python3 tried to read my $PYTHONSTARTUP file, which used
print statements. people will have to run both python 2 and python 3
code at the same time. Using different environment variables will make
this easier.

- Ralf


From ralf at brainbot.com  Wed Jan 13 18:55:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 18:55:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
	(Guido van Rossum's message of "Wed, 13 Jan 2010 09:31:39 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<ca471dc21001130931k6ed40ae1scd6cbbac294d7ae7@mail.gmail.com>
Message-ID: <87k4vldh4c.fsf@muni.brainbot.com>

Guido van Rossum <guido at python.org> writes:

> Somehow the bug site doesn't load for me right now, but I'm -1 on
> this. There are maybe a dozen PYTHON... variables -- we really
> shouldn't try to add PYTHON3 variants for all of them.
>
> Specifically for PYTHONPATH, I feel that its use is always a
> short-time or localized hack, not something you set in your .profile
> nd forget about it (forgetting about it inparticular often leads to
> unnecessary debugging pain later). The better approaches are based on
> site-packages, wrapper scripts, virtualenv, etc.

then at least remove them completely from python3, so one can use them
for his python 2 installation. Otherwise it will stump on some innocent
python 3 program (and cause unnecessary debugging pain).



From steven.bethard at gmail.com  Wed Jan 13 18:57:29 2010
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 13 Jan 2010 09:57:29 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:40 AM, Ralf Schmitt <ralf at brainbot.com> wrote:
> "R. David Murray" <rdmurray at bitdance.com> writes:
>
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable. ?Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3. ?It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>>
>> [1] http://bugs.python.org/issue2375
>
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

How complicated is your PYTHONSTARTUP file? My suspicion is that you
could easily write it to work for both Python 2.X and 3.X.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus


From ssteinerx at gmail.com  Wed Jan 13 18:57:42 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 13 Jan 2010 12:57:42 -0500
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>


On Jan 13, 2010, at 12:40 PM, Ralf Schmitt wrote:

> "R. David Murray" <rdmurray at bitdance.com> writes:
> 
>> Please review issue 2375 [1], which is an enhancement request to add a
>> PYTHON3PATH environment variable.  Because we have elected to have both
>> a python and a python3 command, I think this is an issue worth thinking
>> about carefully to make sure we are serving the Python user community
>> and easing the transition to python3.  It could be that "use virtualenv"
>> is the best answer, but I feel we should think about it carefully to
>> make sure that is really true.
>> 
>> [1] http://bugs.python.org/issue2375
> 
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

Or, how about just removing the antiquated use of environment variables altogether from Python 3 and avoid the issue completely.  

Python 2 will continue to run as it has, and better solutions will emerge for Python 3 development.

Beats the heck out of making a whole duplicate set for PYTHON3BLAHBLAH, yuck!

S



From guido at python.org  Wed Jan 13 18:58:04 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 09:58:04 -0800
Subject: [Python-Dev] regex module
In-Reply-To: <bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
References: <4B4CF354.2050603@mrabarnett.plus.com>
	<bbaeab101001121958t20a77a4oa27cf1538f7557ac@mail.gmail.com>
Message-ID: <ca471dc21001130958k6edabaacof256b5e811c75ea6@mail.gmail.com>

Memories of days past... Python had several regular expression
implementations before, one of which was called "regex".

But I would rather not have a new module -- I would much rather have a
flag specifying the new (backwards incompatible) syntax/semantics. The
flag would have a long name (e.g. re.NEW_SYNTAX), a short name (e.g.
re.N) and an inline syntax, "(?n)...".

--Guido

On Tue, Jan 12, 2010 at 7:58 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Jan 12, 2010 at 14:10, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> Hi all,
>>
>> I'm back on the regex module after doing other things and I'd like your
>> opinion on a number of matters:
>>
>> Firstly, the current re module has a bug whereby it doesn't split on
>> zero-width matches. The BDFL has said that this behaviour should be
>> retained by default in case any existing software depends on it. My
>> question is: should my regex module still do this for Python 3?
>> Speaking personally, I'd like it to behave correctly, and Python 3 is
>> the version where backwards-compatibility is allowed to be broken.
>>
>
> If it is a separate module under a different name it can do the proper
> thing. People will just need to be aware of the difference when they import
> the module.
>
>>
>> Secondly, Python 2 is reaching the end of the line and Python 3 is the
>> future. Should I still release a version that works with Python 2? I'm
>> thinking that it could be confusing if new regex module did zero-width
>> splits correctly in Python 3 but not in Python 2. And also, should I
>> release it only for Python 3 as a 'carrot'?
>>
>
> That's totally up to you. There is practically no chance of it getting into
> the 2.x under the stdlib at this point since 2.7b1 is coming up and this
> module has not been out in the wild for a year (to my knowledge). ?If you
> want to support 2.x that's fine and I am sure users would appreciate it, but
> it isn't necessary to get into the Python 3 stdlib.
>
>>
>> Finally, the module allows some extra backslash escapes, eg \g<name>, in
>> the pattern. Should it treat ill-formed escapes, eg \g, as it would have
>> treated them in the re module?
>>
>
> If you want to minimize the differences then it should probably match. As I
> said, since it is a different name to import under it can deviate where
> reasonable, just make sure to clearly document the deviations.
> -Brett
>
>>
>> Thanks
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)


From ralf at brainbot.com  Wed Jan 13 19:03:08 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 19:03:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
	(Steven Bethard's message of "Wed, 13 Jan 2010 09:57:29 -0800")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<d11dcfba1001130957m713d3b19vb754dbb195236c2@mail.gmail.com>
Message-ID: <87fx69dgrn.fsf@muni.brainbot.com>

Steven Bethard <steven.bethard at gmail.com> writes:

>
> How complicated is your PYTHONSTARTUP file? My suspicion is that you
> could easily write it to work for both Python 2.X and 3.X.

sure. that's exactly what I did. My point is that sharing those
environment variables will cause pain for some people.


From guido at python.org  Wed Jan 13 19:07:58 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:07:58 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>

On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
just removing the antiquated use of environment variables altogether
from Python 3 and avoid the issue completely.

-1. They have their use, but more in controlled situations. If you
have "global" env vars that you only want to use with Python 2.x,
write a wrapper for Python 3 that invokes it with -E.

-- 
--Guido van Rossum (python.org/~guido)


From scott+python-dev at scottdial.com  Wed Jan 13 19:14:21 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 13 Jan 2010 13:14:21 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E0D7D.3040806@scottdial.com>

On 1/13/2010 9:04 AM, Nick Coghlan wrote:
> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

That is not correct. There are IA-64 versions of Window Server.

>From [1]:
"Backward compatibility is a key point differentiating Itanium from x86
and x64 architectures."

So to echo what Michael said, the Microsoft nomenclature is "x64"
regardless of yours and Martin's objections to that name. Nobody who
uses Windows would be confused by "x64" since that is *the* Microsoft
naming scheme. And, anyone using IA64 will expect to see "IA64" and not
"x64", and furthermore anyone using IA64 is likely aware of what
processor they are using (being as there are only Windows Server
editions for IA64) and the difference between "IA64" and "x64".

I don't think an end-user facing page is a great place for pedantic and
wordy defenses of defying the Microsoft-blessed naming scheme.

> Linux, on the other hand, can run on multiple 64 bit architectures, so
> being more specific and using the official AMD64 nomenclature makes
> sense.

This has no relevance to the conversation since there are no Linux
binaries being distributed. The conversation on the expectations of
Windows end-users, who are the target of the download links.

[1] http://www.microsoft.com/servers/64bit/itanium/overview.mspx

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From guido at python.org  Wed Jan 13 19:22:33 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:22:33 -0800
Subject: [Python-Dev] topics I plan to discuss at the language summit
In-Reply-To: <loom.20100113T131816-515@post.gmane.org>
References: <bbaeab101001101225k788dae4fo3e50511b2e53a86d@mail.gmail.com> 
	<loom.20100113T131816-515@post.gmane.org>
Message-ID: <ca471dc21001131022w429e4ceal899235bc711ecaca@mail.gmail.com>

On Wed, Jan 13, 2010 at 4:24 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> If there is something missing from the stdlib discussion that you think should
> be brought up at the summit let me know. And if there is something here you want
> to discuss before the summit let me know and I can start a separate thread on it.
>
> I'm sorry I won't be able to come to the language summit, but I would like if
> possible to expedite a pronouncement on PEP 391 (configuring logging using
> dictionaries). I believe I addressed all the comments made on the discussion
> threads mentioned in the PEP and so I'm not sure what more I need to do to get a
> pronouncement. I guess the stdlib slot gives an opportunity for people to air
> their views and so I'd be grateful if you added it to the agenda.

Is the PEP in the stage of consensus yet? If so it may not even be
necessary to discuss it at the summit (though I certainly won't stop
people if they want to).

-- 
--Guido van Rossum (python.org/~guido)


From phd at phd.pp.ru  Wed Jan 13 19:18:50 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 21:18:50 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
Message-ID: <20100113181850.GA3837@phd.pp.ru>

On Wed, Jan 13, 2010 at 12:57:42PM -0500, ssteinerX at gmail.com wrote:
> Or, how about just removing the antiquated use of environment variables

   "antiquated"? You are kidding, aren't you?!

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From guido at python.org  Wed Jan 13 19:51:14 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 10:51:14 -0800
Subject: [Python-Dev] PyCon Keynote
Message-ID: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>

Please mail me topics you'd like to hear me talk about in my keynote
at PyCon this year.

-- 
--Guido van Rossum (python.org/~guido)


From martin at v.loewis.de  Wed Jan 13 20:13:58 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:13:58 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4D9310.6060907@voidspace.org.uk>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de> <4B4D9310.6060907@voidspace.org.uk>
Message-ID: <4B4E1B76.4010309@v.loewis.de>

>>>   * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>> X86-64 binary -- does not include source)
>>>
>>> instead of:
>>>
>>>   * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>> not include source)
>>>      
>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>> to use AMD64 instead. I think we should comply - they invented the
>> architecture, so they have the right to give it a name. Neither
>> Microsoft nor Intel have such a right.
>>    
> 
> I think we should use whatever is most informative and least confusing
> to our users - we owe our allegiance to them and not to a processor vendor.

And why do you think this is x86-64?

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:33:24 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:33:24 +0100
Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2
In-Reply-To: <4B4DA0DA.7070007@gmail.com>
References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com>	<hid9s1$i05$1@ger.gmane.org>	<4B4A373F.9050601@gmail.com>	<4B4A5DCE.3070909@v.loewis.de>	<DB9A0210-0515-457D-A7FC-AB7065BB9970@python.org>	<4B4B9B55.1010709@v.loewis.de>	<20100111191128.39020d89@freewill>	<4B4CEF3D.8000107@v.loewis.de>
	<4B4CF17A.1000503@voidspace.org.uk> <4B4DA0DA.7070007@gmail.com>
Message-ID: <4B4E2004.8050905@v.loewis.de>

> Note that increased 3.x compatibility in the most recent 2.x release
> will always help in two scenarios:
> 
> 1. New projects that want to use 2.x only libraries but want to be ready
> for the Py3k transition in their own code (e.g. new 2.7 features like
> set literals, dict and set comprehensions and the view* dict methods can
> be translated far more cleanly by 2to3 than the closest comparable 2.6
> code).

Of these, I can only see this being the case of the view* methods.

Why would set literals allow for code that more cleanly translates
to 3.x? Suppose you write

 f = set((1,2,3))

in 2.6. That code will work *exactly* the same way in 3.1, no
translation needed at all. Likewise for dict and set comprehensions:
the code is as cleanly translated in the 2.7 form as it is in any
prior form (ie. no modifications).

As for view* methods: there is one important exception where
the 2.6 equivalent is as cleanly translated as a view method:

for key in D:
  action

This is a more modern equivalent for iterating over D.keys(),
so if your code still does the latter, just change it to the
former (requires Python 2.2). I'd claim that this is the dominant
case of traversing a dictionary (probably followed by iterating
over .items()). So while it is true that only view* can be
transformed correctly, I'd claim that this is an infrequent
issue.

> 2. Similarly new projects that use a 3.x trunk can be more readily
> translated to a 2.7 version with 3to2, whereas some constructs may be
> difficult to recreate in earlier Python versions.

That may be true, but is besides the point here: the issue was whether
a 2.8 release might help in migrating to 3.x. People who follow this
approach already have 3.x code. Whether they would rather port to 2.8
only, and get that work with little effort, or whether they would port
to 2.5 and later, and put in more work, I don't know - I guess we would
have to ask these people. I would expect that they prefer if 2.x
died rather sooner than later, because they can then stop supporting
it.

Regards,
Martin



From tjreedy at udel.edu  Wed Jan 13 20:36:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 Jan 2010 14:36:03 -0500
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <hil7av$52r$1@ger.gmane.org>

On 1/13/2010 2:13 PM, "Martin v. L?wis" wrote:

>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>
> And why do you think this is x86-64?

I do not believe I have personally seen, or at least noticed, that 
specific term before. Something like

"Release for 64-bit Windows (Win NT, 2000, XP, Vista, 7 <or whatever 
correct list is>) on AMD64-type processors from AMD and Intel"

should be clear enough.




From martin at v.loewis.de  Wed Jan 13 20:40:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 13 Jan 2010 20:40:30 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4DD2EC.3030709@gmail.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com>
Message-ID: <4B4E21AE.40602@v.loewis.de>

> As Windows doesn't run on non-x86 architectures, their naming is
> generally just Windows <Whatever> (32 bit) and Windows <Whatever> (64 bit).

Windows actually does - it runs on IA-64 (which is a non-x86 architecture).

However, end users are unlikely to use such hardware, so distinguishing
between 32-bit and 64-bit is typically good enough.

> In this case, making it clear that the 64-bit Windows binaries work for
> both Intel and AMD chips is more important than using only the official
> architecture name.

Well, we did have Itanium binaries at one point, and the "64-bit Windows
binaries" you are referring to most definitely don't run on Itanium,
even though that's an Intel chip.

Regards,
Martin


From martin at v.loewis.de  Wed Jan 13 20:45:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 20:45:40 +0100
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E0D7D.3040806@scottdial.com>
References: <4B4CFBD9.1090009@voidspace.org.uk>	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>	<4B4DD2EC.3030709@gmail.com>
	<4B4E0D7D.3040806@scottdial.com>
Message-ID: <4B4E22E4.4040504@v.loewis.de>

> So to echo what Michael said, the Microsoft nomenclature is "x64"
> regardless of yours and Martin's objections to that name. Nobody who
> uses Windows would be confused by "x64" since that is *the* Microsoft
> naming scheme.

That's actually not entirely true. There are several places in the
APIs where Microsoft either allows or requires to call the architecture
AMD64 (e.g. architecture indication in MSI files).

Regards,
Martin


From regebro at gmail.com  Wed Jan 13 20:50:59 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 20:50:59 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87r5ptdhu3.fsf@muni.brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
Message-ID: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>

On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
> The first thing I got while trying to run a python3 prompt few days ago,
> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
> print statements. people will have to run both python 2 and python 3
> code at the same time. Using different environment variables will make
> this easier.

What do you need to do in the PYTHONSTARTUP file?
Ten years of Python programming, and I didn't even know it existed. :-)

-- 
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64


From phd at phd.pp.ru  Wed Jan 13 21:08:40 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 13 Jan 2010 23:08:40 +0300
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <20100113200840.GC14858@phd.pp.ru>

On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From murman at gmail.com  Wed Jan 13 21:12:11 2010
From: murman at gmail.com (Michael Urman)
Date: Wed, 13 Jan 2010 14:12:11 -0600
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E22E4.4040504@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk> <4B4D0890.2030505@cheimes.de>
	<4B4D6425.3060307@v.loewis.de>
	<dcbbbb411001130535h7de47a36m335531ef58099b6e@mail.gmail.com>
	<4B4DD2EC.3030709@gmail.com> <4B4E0D7D.3040806@scottdial.com>
	<4B4E22E4.4040504@v.loewis.de>
Message-ID: <dcbbbb411001131212q3ef907c8i67e8c03c96c31e2f@mail.gmail.com>

On Wed, Jan 13, 2010 at 13:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> So to echo what Michael said, the Microsoft nomenclature is "x64"
>> regardless of yours and Martin's objections to that name. Nobody who
>> uses Windows would be confused by "x64" since that is *the* Microsoft
>> naming scheme.
>
> That's actually not entirely true. There are several places in the
> APIs where Microsoft either allows or requires to call the architecture
> AMD64 (e.g. architecture indication in MSI files).

I should have clarified I was talking about the names shown on MSDN
subscriptions for downloading installation media of Windows 7 and
Windows Vista. It's pretty clear in that context Microsoft uses x64 to
describe this platform of Windows. But again, it's far from clear that
this is a term they use for non-developers.

-- 
Michael Urman


From regebro at gmail.com  Wed Jan 13 21:27:08 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 13 Jan 2010 21:27:08 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <20100113200840.GC14858@phd.pp.ru>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<20100113200840.GC14858@phd.pp.ru>
Message-ID: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>

On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>> What do you need to do in the PYTHONSTARTUP file?
>> Ten years of Python programming, and I didn't even know it existed. :-)
>
> ? See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.

Cheese and fries! :-)

Anyway, I don't see how this could pose any problems, because any user
advanced enough to do these things will be advanced enough to
understand what the problem is and fix it so it works under Python 3
too.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64


From ralf at brainbot.com  Wed Jan 13 21:52:34 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Wed, 13 Jan 2010 20:52:34 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	(Lennart Regebro's message of "Wed, 13 Jan 2010 20:50:59 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <874ompg225.fsf@brainbot.com>

Lennart Regebro <regebro at gmail.com> writes:

> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
>
> What do you need to do in the PYTHONSTARTUP file?
> Ten years of Python programming, and I didn't even know it existed. :-)

hehe. tab completion:

# -*- mode: python -*- 
# Last changed: 2009-12-23 22:25:15 by ralf

import sys
import os

def initreadline():
    
    try:
        import readline
    except ImportError:
        sys.stdout.write("Module readline not available.\n")
        return
    
    import rlcompleter
    readline.parse_and_bind("tab: complete")
    
    # Use tab for completions
    readline.parse_and_bind('tab: complete')
    # This forces readline to automatically print the above list when tab
    # completion is set to 'complete'.
    readline.parse_and_bind('set show-all-if-ambiguous on')
    # Bindings for incremental searches in the history. These searches
    # use the string typed so far on the command line and search
    # anything in the previous input history containing them.
    readline.parse_and_bind('"\C-r": reverse-search-history')
    readline.parse_and_bind('"\C-s": forward-search-history')

    history = os.path.expanduser("~/.pyhistory") 
    if os.path.exists(history):
        readline.read_history_file(history)
        
    import atexit
    atexit.register(lambda: readline.write_history_file(history))
    
initreadline()
del initreadline


From martin at v.loewis.de  Wed Jan 13 22:05:03 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 13 Jan 2010 22:05:03 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <4B4E357F.4050107@v.loewis.de>

Lennart Regebro wrote:
> On Wed, Jan 13, 2010 at 18:40, Ralf Schmitt <ralf at brainbot.com> wrote:
>> The first thing I got while trying to run a python3 prompt few days ago,
>> was an error. python3 tried to read my $PYTHONSTARTUP file, which used
>> print statements. people will have to run both python 2 and python 3
>> code at the same time. Using different environment variables will make
>> this easier.
> 
> What do you need to do in the PYTHONSTARTUP file?

I personally set up readline: set tab completion, and load the history
of commands from the previous session.

Of course, I don't know what Ralf uses it for.

Regards,
Martin


From ncoghlan at gmail.com  Wed Jan 13 22:43:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 07:43:41 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
Message-ID: <4B4E3E8D.2010407@gmail.com>

Guido van Rossum wrote:
> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
> just removing the antiquated use of environment variables altogether
> from Python 3 and avoid the issue completely.
> 
> -1. They have their use, but more in controlled situations. If you
> have "global" env vars that you only want to use with Python 2.x,
> write a wrapper for Python 3 that invokes it with -E.

Perhaps a case can be made for Python 3 to assume -E by default, with a
-e option to enable reading of the environment variables?

That way naive users could run Python3 without tripping over existing
Python2 environment variables, while other tools could readily set up a
different environment before launching Python3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From guido at python.org  Wed Jan 13 23:45:57 2010
From: guido at python.org (Guido van Rossum)
Date: Wed, 13 Jan 2010 14:45:57 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net> 
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com> 
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com> 
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <ca471dc21001131445g639c6867ude83f03a77eb72b9@mail.gmail.com>

On Wed, Jan 13, 2010 at 1:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
>
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
>
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users using environment variables? That's a recipe for disaster
in any version! :-)

-- 
--Guido van Rossum (python.org/~guido)


From jared.grubb at gmail.com  Wed Jan 13 23:56:37 2010
From: jared.grubb at gmail.com (Jared Grubb)
Date: Wed, 13 Jan 2010 14:56:37 -0800
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <13F6C43A-D62A-4A9F-AF7A-9BDAC94F7D72@gmail.com>

On 13 Jan 2010, at 13:43, Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>> 
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

I raised a question about these PYTHON3* variables once before in a discussion about shebang lines:
http://www.mailinglistarchive.com/python-dev at python.org/msg29500.html

I'm not advocating them, but just wanted to make sure to bring up the shebang use case.

Jared

From skip at pobox.com  Wed Jan 13 23:24:13 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 16:24:13 -0600
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
Message-ID: <19278.18445.428716.321365@montanaro.dyndns.org>


    Lennart> What do you need to do in the PYTHONSTARTUP file?

Just reading off stuff from my own personal startup file...  I use it for
stuff I want available during interactive sessions:

    1. Enable true division.

    2. Conditionally define "help" from back in the days when there was no
       help builtin function.

    3. Auto-save session (readline) history so I can easily recall commands
       across sessions.

    4. Add other fake builtins ("like") or override behavior of some (like
       "dir") making them handier for interactive use.

    5. autoload modules/symbols (pokes around in common modules from
       sys.excepthook function).

Oh, and I've had no particular trouble keeping it working in Python 1, 2 or
3.

Skip




From fuzzyman at voidspace.org.uk  Thu Jan 14 01:20:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 00:20:21 +0000
Subject: [Python-Dev] Fwd: Download Page - AMD64
In-Reply-To: <4B4E1B76.4010309@v.loewis.de>
References: <4B4CFBD9.1090009@voidspace.org.uk>
	<4B4D0890.2030505@cheimes.de>	<4B4D6425.3060307@v.loewis.de>
	<4B4D9310.6060907@voidspace.org.uk> <4B4E1B76.4010309@v.loewis.de>
Message-ID: <4B4E6345.5070202@voidspace.org.uk>

On 13/01/2010 19:13, "Martin v. L?wis" wrote:
>>>>    * Python 2.6.4 Windows X86-64 installer (Windows AMD64 / Intel 64 /
>>>> X86-64 binary -- does not include source)
>>>>
>>>> instead of:
>>>>
>>>>    * Python 2.6.4 Windows AMD64 installer (Windows AMD64 binary -- does
>>>> not include source)
>>>>
>>>>          
>>> -1. AMD doesn't want us to use the term x86-64 anymore, but wants us
>>> to use AMD64 instead. I think we should comply - they invented the
>>> architecture, so they have the right to give it a name. Neither
>>> Microsoft nor Intel have such a right.
>>>
>>>        
>> I think we should use whatever is most informative and least confusing
>> to our users - we owe our allegiance to them and not to a processor vendor.
>>      
> And why do you think this is x86-64?
>    

Well anecdotal everyone I have *every* talked to about 64bit processors 
has referred to having a 64bit processor (x86 is a given) and not an 
AMD64 architecture processor.

Linus Torvalds addressed this specific issue for Linux and came down on 
the side of "x86-64": http://kerneltrap.org/node/2466
Look up AMD64 on Wikipedia and it redirects you to the X86-64 page.
Information website setup by AMD and partners about the AMD64 
architecture: http://www.x86-64.org/about.html
In the AMD website they refer to "x86-64 Assembly": 
http://www.x86-64.org/documentation/assembly.html

Microsoft seem to draw a distinction between x64 (which would also be 
acceptable) and Itanium based systems. Very rarely do they refer to AMD64:

* http://www.microsoft.com/servers/64bit/compare.mspx
* http://www.microsoft.com/servers/64bit/x64/overview.mspx
* http://www.microsoft.com/servers/64bit/overview.mspx

Using a vendor specific name automatically begs the question as to 
whether the installer works on processors from other vendors, as we saw 
in the specific enquiry from the user that triggered this debate.

Referring to the AMD 64 build as x86-64, with a footnote explaining 
which architectures this specifically means is unlikely to confuse 
people. It is *definitely* better than just saying AMD64.

All the best,

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From skip at pobox.com  Thu Jan 14 02:50:55 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 13 Jan 2010 19:50:55 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
Message-ID: <19278.30847.649228.115514@montanaro.dyndns.org>


Just out of curiosity, is the static build stuff (use the old Modules/Setup
file to build modules) exercised at all any more?

Skip



From benjamin at python.org  Thu Jan 14 03:22:03 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 13 Jan 2010 20:22:03 -0600
Subject: [Python-Dev] static (Modules/Setup) builds?
In-Reply-To: <19278.30847.649228.115514@montanaro.dyndns.org>
References: <19278.30847.649228.115514@montanaro.dyndns.org>
Message-ID: <1afaf6161001131822r151bd202x35653ba44ee0235d@mail.gmail.com>

2010/1/13  <skip at pobox.com>:
>
> Just out of curiosity, is the static build stuff (use the old Modules/Setup
> file to build modules) exercised at all any more?

Exercised as in used in the build process? Yes.


-- 
Regards,
Benjamin


From vinay_sajip at yahoo.co.uk  Thu Jan 14 10:23:47 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 09:23:47 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
Message-ID: <954450.26617.qm@web25804.mail.ukl.yahoo.com>

In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:

  http://www.python.org/dev/peps/pep-0391/

In November 2009 I posted to this list that the PEP was ready for review.

I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.

So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.

Thanks & regards,

Vinay Sajip



      



From ziade.tarek at gmail.com  Thu Jan 14 10:30:15 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 14 Jan 2010 10:30:15 +0100
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <94bdd2611001140130u6154e893jfd37db32a25be66a@mail.gmail.com>

On Thu, Jan 14, 2010 at 10:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
[..]
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would
> be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check
> in the changes unless there are objections to doing so. All those who think that logging is currently
> hard to configure will benefit from these changes, so here's the opportunity to pipe up and
> improve things.


FWIW, I am +1. Thanks for your work

Tarek


From hodgestar+pythondev at gmail.com  Thu Jan 14 10:39:41 2010
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 14 Jan 2010 11:39:41 +0200
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <fb73205e1001140139n4b920871t6bfc6b91c469264f@mail.gmail.com>

On Thu, Jan 14, 2010 at 11:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I think the dict configuration scheme will be a great addition to the
logging module. :)

Schiavo
Simon


From p.f.moore at gmail.com  Thu Jan 14 12:22:09 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 14 Jan 2010 11:22:09 +0000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>

2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> So, can you please indicate your vote for or against incorporating PEP 391 into Python?

I've no immediate need for the feature, but it would be good to have
something like this, so I'm +1.
Paul.


From ncoghlan at gmail.com  Thu Jan 14 12:46:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:46:19 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<79990c6b1001140322j47fe2030ree2f1cebbc169108@mail.gmail.com>
Message-ID: <4B4F040B.8010607@gmail.com>

Paul Moore wrote:
> 2010/1/14 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> So, can you please indicate your vote for or against incorporating PEP 391 into Python?
> 
> I've no immediate need for the feature, but it would be good to have
> something like this, so I'm +1.

I'm in the same boat as Paul, but PEP 291 provides a nice forward
compatible configuration scheme that will work with any application
configuration approach that can produce an appropriate Python dictionary.

So +1 from me too - I think the PEP has now taken this as far as
theorising can go, and the only way to learn anything further is to put
it into practice.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Jan 14 12:57:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 14 Jan 2010 21:57:55 +1000
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4B4F06C3.7030806@gmail.com>

Vinay Sajip wrote:
> In October 2009 I created PEP 391 to propose a new method of
> configuring logging using dictionaries:
> 
> http://www.python.org/dev/peps/pep-0391/

Although one minor comment: you can probably remove the note about the
"ext://" and "cfg://" prefixes being provisional at this stage. Those
names look fine to me, so I don't see any point inviting a late-breaking
bikeshed discussion about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From jnoller at gmail.com  Thu Jan 14 14:53:29 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 08:53:29 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>

On Thu, Jan 14, 2010 at 4:23 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In October 2009 I created PEP 391 to propose a new method of configuring logging using dictionaries:
>
> ?http://www.python.org/dev/peps/pep-0391/
>
> In November 2009 I posted to this list that the PEP was ready for review.
>
> I have had numerous helpful comments from some of you, and I have incorporated most of them into updates to the PEP. Though I have the feeling from community discussions that the PEP has been generally favourably received - well I would think that, wouldn't I? ;-) - I've not asked for a vote and so I don't know the state of community consensus regarding this PEP.
>
> So, can you please indicate your vote for or against incorporating PEP 391 into Python? It would be nice if I could incorporate the changes before 2.7a3 is released! Ideally, I would like to check in the changes unless there are objections to doing so. All those who think that logging is currently hard to configure will benefit from these changes, so here's the opportunity to pipe up and improve things.
>
> Thanks & regards,
>
> Vinay Sajip

I'm generally +1 - but given I know that Django 1.2 is slated to
implement something somewhat similar, I'm interested to hear how this
proposal meshes with their plan(s).

jesse


From vinay_sajip at yahoo.co.uk  Thu Jan 14 15:08:24 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 14 Jan 2010 14:08:24 +0000 (GMT)
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
Message-ID: <92210.91656.qm@web25806.mail.ukl.yahoo.com>

> From: Jesse Noller <jnoller at gmail.com>

> I'm generally +1 - but given I know that Django 1.2 is slated to
> implement something somewhat similar, I'm interested to hear how this
> proposal meshes with their plan(s)..

Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:

http://groups.google.com/group/django-developers/msg/4ef81a2160257221

They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).

Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

Regards,

Vinay Sajip


      



From ssteinerx at gmail.com  Thu Jan 14 15:54:27 2010
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Thu, 14 Jan 2010 09:54:27 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <62E362ED-5DD7-4D42-B5E0-B263A137FD5C@gmail.com>


On Jan 14, 2010, at 9:08 AM, Vinay Sajip wrote:

>> From: Jesse Noller <jnoller at gmail.com>
> 
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
> 
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
> 
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
> 
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
> 
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.

That puts a huge +1 on there for me.  

If we can get this approved and have Django's logging improved *and* avoid a whole pile of duplicate effort in Django that's a huge win.

S



From jnoller at gmail.com  Thu Jan 14 15:56:18 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 14 Jan 2010 09:56:18 -0500
Subject: [Python-Dev] PEP 391 - Please Vote!
In-Reply-To: <92210.91656.qm@web25806.mail.ukl.yahoo.com>
References: <954450.26617.qm@web25804.mail.ukl.yahoo.com>
	<4222a8491001140553o5b91af27sd0e8402bcfca49e7@mail.gmail.com>
	<92210.91656.qm@web25806.mail.ukl.yahoo.com>
Message-ID: <4222a8491001140656w68c7290agccfe7282cf96f2fb@mail.gmail.com>

On Thu, Jan 14, 2010 at 9:08 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> From: Jesse Noller <jnoller at gmail.com>
>
>> I'm generally +1 - but given I know that Django 1.2 is slated to
>> implement something somewhat similar, I'm interested to hear how this
>> proposal meshes with their plan(s)..
>
> Django 1.2 will most likely not implement logging - they're now in feature freeze for big features. See Russ Magee's post:
>
> http://groups.google.com/group/django-developers/msg/4ef81a2160257221
>
> They're waiting for a Python decision and (from Russ Magee's post) seem to be in favour of the changes proposed in PEP 391. If we get these changes into Python soon, then Django 1.3 may be able to make use of them in its logging (which encompasses not just configuring Django logging in a settings.py, but also using logging internally throughout Django where it makes sense to do so).
>
> Assuming PEP 391 gets the nod, then after implementing the changes into Python, I plan to work with the Django community to get improved logging support in Django for 1.3.
>
> Regards,
>
> Vinay Sajip
>

Cool, +1 then :)


From mal at egenix.com  Thu Jan 14 20:19:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 20:19:12 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4E3E8D.2010407@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com>
Message-ID: <4B4F6E30.50400@egenix.com>

Nick Coghlan wrote:
> Guido van Rossum wrote:
>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>> just removing the antiquated use of environment variables altogether
>> from Python 3 and avoid the issue completely.
>>
>> -1. They have their use, but more in controlled situations. If you
>> have "global" env vars that you only want to use with Python 2.x,
>> write a wrapper for Python 3 that invokes it with -E.
> 
> Perhaps a case can be made for Python 3 to assume -E by default, with a
> -e option to enable reading of the environment variables?
> 
> That way naive users could run Python3 without tripping over existing
> Python2 environment variables, while other tools could readily set up a
> different environment before launching Python3.

Naive users won't have PYTHONPATH or any other Python environment
variables setup.

Really, if you know that you are going to run Python 3 instead of
Python 2 or vice-versa it's easy enough to run

. py3env.sh
or
. py2env.sh

in order to setup your shell environment, much like you typically
do when using virtual Python installations or work on different
projects that require different setups.

If you just want to separate Python 2 and 3 files, use the
user site-packages dir which includes the Python version.

More experienced users could also write their own environment
switching sitecustomize.py or usercustomize.py.

And then there's the hackery option for experts that love
dirty tricks: add a .pth file which includes an "import mysetup"
line to fix-up you path and other settings.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From ncoghlan at gmail.com  Thu Jan 14 22:02:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 15 Jan 2010 07:02:28 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <4B4F8664.4080107@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Guido van Rossum wrote:
>>> On Wed, Jan 13, 2010 at 9:57 AM, ssteinerX at gmail.com > Or, how about
>>> just removing the antiquated use of environment variables altogether
>>> from Python 3 and avoid the issue completely.
>>>
>>> -1. They have their use, but more in controlled situations. If you
>>> have "global" env vars that you only want to use with Python 2.x,
>>> write a wrapper for Python 3 that invokes it with -E.
>> Perhaps a case can be made for Python 3 to assume -E by default, with a
>> -e option to enable reading of the environment variables?
>>
>> That way naive users could run Python3 without tripping over existing
>> Python2 environment variables, while other tools could readily set up a
>> different environment before launching Python3.
> 
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.

I was actually thinking of the situation where the OS had a preinstalled
Python 2 with a default PYTHONPATH setting and the user was playing with
Python 3.

However, I agree that that is a fairly unlikely scenario (since
preinstalled Pythons tend not to rely on the environment variables and a
user sophisticated enough to be playing with a new Python interpreter
should be able to cope with a few environment variable conflicts anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From fuzzyman at voidspace.org.uk  Thu Jan 14 22:09:30 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 14 Jan 2010 21:09:30 +0000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F8664.4080107@gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <4B4F8664.4080107@gmail.com>
Message-ID: <4B4F880A.5010809@voidspace.org.uk>

On 14/01/2010 21:02, Nick Coghlan wrote:
> However, I agree that that is a fairly unlikely scenario (since
> preinstalled Pythons tend not to rely on the e
Well, on the other hand I think that during the next few years it will 
be increasingly common for developers (and possibly users) to have 
Python 2 and Python 3 installed side-by-side.

Many libraries and applications may never make the jump to Python 3 and 
Python users may be using 'legacy' Python 2 code for many years to come. 
It will also become increasingly common for developers to be using 
Python 3 *primarily* and for Python 3 only libraries and applications to 
emerge.

Whilst there are workarounds we *are* in a situation that Python 2 and 
Python 3 share environment variables for the location of libraries and 
executing code on startup, whilst at the same time they are largely 
incompatible and need separate library paths and startup code.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ralf at brainbot.com  Thu Jan 14 22:25:31 2010
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 14 Jan 2010 22:25:31 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <4B4F6E30.50400@egenix.com> (M.'s message of "Thu, 14 Jan 2010
	20:19:12 +0100")
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>
	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>
	<4B4E3E8D.2010407@gmail.com> <4B4F6E30.50400@egenix.com>
Message-ID: <871vhswf90.fsf@brainbot.com>

"M.-A. Lemburg" <mal at egenix.com> writes:

>
> Naive users won't have PYTHONPATH or any other Python environment
> variables setup.
>
> Really, if you know that you are going to run Python 3 instead of
> Python 2 or vice-versa it's easy enough to run

You don't even know that you're going to run python. I have 40 python
scripts in my /usr/bin directory. 

>
> . py3env.sh
> or
> . py2env.sh
>
> in order to setup your shell environment, much like you typically
> do when using virtual Python installations or work on different
> projects that require different setups.

No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.

>
> If you just want to separate Python 2 and 3 files, use the
> user site-packages dir which includes the Python version.

Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
install those programs in a user's home directory. There are still
people running python <2.6.


From mal at egenix.com  Thu Jan 14 22:51:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 14 Jan 2010 22:51:07 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <871vhswf90.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<0E5A3C2E-1134-40A3-8735-C8C72A685811@gmail.com>	<ca471dc21001131007l69e1da70p3c9fbf1b40188683@mail.gmail.com>	<4B4E3E8D.2010407@gmail.com>
	<4B4F6E30.50400@egenix.com> <871vhswf90.fsf@brainbot.com>
Message-ID: <4B4F91CB.2060106@egenix.com>

Ralf Schmitt wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>
>> Naive users won't have PYTHONPATH or any other Python environment
>> variables setup.
>>
>> Really, if you know that you are going to run Python 3 instead of
>> Python 2 or vice-versa it's easy enough to run
> 
> You don't even know that you're going to run python. I have 40 python
> scripts in my /usr/bin directory. 
> 
>>
>> . py3env.sh
>> or
>> . py2env.sh
>>
>> in order to setup your shell environment, much like you typically
>> do when using virtual Python installations or work on different
>> projects that require different setups.
> 
> No, thanks. I'd rather setup my shell environment in ~/.zshrc, once.
> 
>>
>> If you just want to separate Python 2 and 3 files, use the
>> user site-packages dir which includes the Python version.
> 
> Both the bzr and mercurial wiki recommend setting PYTHONPATH in order to
> install those programs in a user's home directory. There are still
> people running python <2.6.

Note that you only need to have a different PYTHONPATH
setups for Python 2.x/3.x if you plan to run code that:

 * is not installed in site-packages (most OS shipped code
   will be found in the resp. system site-packages/ dir)

 * is not installed in a user site-packages directory
   (user installed code will typically go there (*))

 * uses modules/packages that come in two different versions
   (one for Python 2.x and one for 3.x) and use the same name
   for both versions

 * needs to work in both Python 2.x and 3.x


(*) The method for installing code in user site-packages dir is
running:

    python setup.py install --user

instead of the standard

    python setup.py install

which install to the system-wide site-packages.

BTW: I guess the bzr and mercurial wikis will need to be updated
accordingly - at least for users of Python >=2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From martin at v.loewis.de  Thu Jan 14 22:55:04 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Jan 2010 22:55:04 +0100
Subject: [Python-Dev] [ANN] Python 2.5.5 Release Candidate 1.
Message-ID: <4B4F92B8.30806@v.loewis.de>

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.5.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.6 (which is
2.6.4 at this point).

This releases fixes issues with the logging and tarfile modules, and
with thread-local variables. See the detailed release notes at the
website (also available as Misc/NEWS in the source distribution) for
details of bugs fixed.

For more information on Python 2.5.5, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.5.5

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

    http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
martin at v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)


From status at bugs.python.org  Fri Jan 15 18:07:26 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Jan 2010 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20100115170726.9963A7865F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (01/08/10 - 01/15/10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2536 open (+32) / 16993 closed (+16) / 19529 total (+48)

Open issues with patches:  1024

Average duration of open issues: 710 days.
Median duration of open issues: 469 days.

Open Issues Breakdown
   open  2501 (+32)
pending    34 ( +0)

Issues Created Or Reopened (53)
_______________________________

PYTHON3PATH environment variable to supersede PYTHONPATH for mul 01/13/10
CLOSED http://bugs.python.org/issue2375    reopened r.david.murray                
                                                                               

Mark the compiler package as deprecated                          01/13/10
       http://bugs.python.org/issue6837    reopened ezio.melotti                  
                                                                               

test_distutils failure                                           01/15/10
CLOSED http://bugs.python.org/issue6961    reopened flox                          
       buildbot                                                                

test_urllib: unsetting missing 'env' variable                    01/08/10
CLOSED http://bugs.python.org/issue7026    reopened flox                          
                                                                               

Two float('nan') are not equal                                   01/08/10
CLOSED http://bugs.python.org/issue7660    created  jmfauth                       
                                                                               

compiling ctypes fails with non-ascii path                       01/08/10
       http://bugs.python.org/issue7661    created  pitrou                        
       patch                                                                   

time.utcoffset()                                                 01/09/10
       http://bugs.python.org/issue7662    created  techtonik                     
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points  01/10/10
CLOSED http://bugs.python.org/issue7663    created  exarkun                       
                                                                               

python logger does not handle IOError Exception                  01/10/10
CLOSED http://bugs.python.org/issue7664    created  yateenjoshi                   
                                                                               

test_urllib2 and test_ntpath fail if path contains "\"           01/10/10
       http://bugs.python.org/issue7665    created  flox                          
                                                                               

test_lib2to3 fails if path contains space                        01/10/10
CLOSED http://bugs.python.org/issue7666    created  flox                          
                                                                               

test_doctest fails with non-ascii path                           01/10/10
       http://bugs.python.org/issue7667    created  flox                          
       buildbot                                                                

test_httpservers fails with non-ascii path                       01/10/10
       http://bugs.python.org/issue7668    created  flox                          
       buildbot                                                                

test_unicode_file fails with non-ascii path                      01/10/10
CLOSED http://bugs.python.org/issue7669    created  flox                          
                                                                               

_sqlite3: Block *all* operations on a closed Connection object   01/10/10
       http://bugs.python.org/issue7670    created  haypo                         
       patch                                                                   

test_popen fails if path contains special char like ";"          01/11/10
       http://bugs.python.org/issue7671    reopened flox                          
       patch                                                                   

_ssl module causes segfault                                      01/10/10
       http://bugs.python.org/issue7672    created  ssoria                        
       patch                                                                   

audioop: check that length is a multiple of the size             01/11/10
       http://bugs.python.org/issue7673    created  haypo                         
       patch                                                                   

select.select() corner cases: duplicate fds, out-of-range fds    01/11/10
       http://bugs.python.org/issue7674    created  cdleary                       
                                                                               

help() doesn't accept unicode literals in built in docstrings    01/11/10
       http://bugs.python.org/issue7675    created  psd                           
       patch                                                                   

IDLE shell shouldn't use TABs                                    01/11/10
       http://bugs.python.org/issue7676    created  cben                          
                                                                               

distutils, better error message for setup.py upload -sign withou 01/11/10
       http://bugs.python.org/issue7677    created  illume                        
                                                                               

subprocess.Popen pipeline example code in the documentation is l 01/11/10
       http://bugs.python.org/issue7678    created  steven.k.wong                 
                                                                               

Warning building 2.7 on OS X 10.6 libintl.h "Present But Cannot  01/12/10
       http://bugs.python.org/issue7679    created  ssteinerX                     
                                                                               

pythonw crash while attempting to start() a thread object        01/12/10
       http://bugs.python.org/issue7680    created  dontbugme                     
                                                                               

Incorrect division in [wave.py]                                  01/12/10
CLOSED http://bugs.python.org/issue7681    created  alfps                         
       patch, needs review                                                     

Optimisation of if with constant expression                      01/12/10
       http://bugs.python.org/issue7682    created  sdefresne                     
       patch                                                                   

Wrong link in HTML documentation                                 01/12/10
CLOSED http://bugs.python.org/issue7683    created  francescor                    
                                                                               

decimal.py: infinity coefficients in tuples                      01/12/10
       http://bugs.python.org/issue7684    created  skrah                         
                                                                               

minor typo in re docs                                            01/12/10
CLOSED http://bugs.python.org/issue7685    created  mikejs                        
       patch                                                                   

redundant open modes 'rbb', 'wbb', 'abb' no longer work on Windo 01/13/10
       http://bugs.python.org/issue7686    created  ivank                         
                                                                               

Bluetooth support untested                                       01/13/10
       http://bugs.python.org/issue7687    created  pitrou                        
                                                                               

TypeError: __name__ must be set to a string object               01/13/10
       http://bugs.python.org/issue7688    created  frankmillman                  
                                                                               

Pickling of classes with a metaclass and copy_reg                01/13/10
       http://bugs.python.org/issue7689    created  craigcitro                    
       patch                                                                   

Wrong PEP number in importlib module manual page                 01/13/10
CLOSED http://bugs.python.org/issue7690    created  francescor                    
                                                                               

test_bsddb3 files are not always removed when test fails         01/13/10
CLOSED http://bugs.python.org/issue7691    created  flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero              01/13/10
CLOSED http://bugs.python.org/issue7692    created  Bugger                        
                                                                               

tarfile.extractall can't have unicode extraction path            01/13/10
       http://bugs.python.org/issue7693    created  pbienst                       
                                                                               

DeprecationWarning from distuils.commands.build_ext needs stackl 01/13/10
       http://bugs.python.org/issue7694    created  tseaver                       
       patch                                                                   

missing termios constants                                        01/13/10
       http://bugs.python.org/issue7695    created  conorh                        
                                                                               

Improve Memoryview/Buffer documentation                          01/13/10
       http://bugs.python.org/issue7696    created  flox                          
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes  01/13/10
CLOSED http://bugs.python.org/issue7697    created  flox                          
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame  01/14/10
CLOSED http://bugs.python.org/issue7698    created  exarkun                       
       patch                                                                   

strptime, strftime documentation                                 01/14/10
       http://bugs.python.org/issue7699    created  mikejs                        
       patch                                                                   

"-3" flag does not work anymore                                  01/14/10
CLOSED http://bugs.python.org/issue7700    created  flox                          
       patch                                                                   

fix output string length for binascii.b2a_uu()                   01/14/10
CLOSED http://bugs.python.org/issue7701    created  haypo                         
       patch                                                                   

Wrong order of parameters of _get_socket in SMTP class in smtpli 01/14/10
       http://bugs.python.org/issue7702    created  alito                         
                                                                               

Replace buffer()-->memoryview() in Lib/ctypes/test/              01/14/10
       http://bugs.python.org/issue7703    created  flox                          
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE         01/14/10
CLOSED http://bugs.python.org/issue7704    created  DhaReaL                       
                                                                               

libpython2.6.so is not linked correctly on FreeBSD when threads  01/15/10
       http://bugs.python.org/issue7705    created  aballier                      
       patch, needs review                                                     

Missing #include guards                                          01/15/10
       http://bugs.python.org/issue7706    created  akrpic77                      
       patch                                                                   

multiprocess.Queue operations during import can lead to deadlock 01/15/10
       http://bugs.python.org/issue7707    created  kripken                       
                                                                               

Conversion of longs to bytes and vice-versa.                     01/11/10
       http://bugs.python.org/issue1023290 reopened mark.dickinson                
       patch                                                                   



Issues Now Closed (67)
______________________

segfault in curses when calling redrawwin() before refresh()      825 days
       http://bugs.python.org/issue1266    mark.dickinson                
                                                                               

"RuntimeError: dictionary changed size during iteration" in Tkin  751 days
       http://bugs.python.org/issue1658    flox                          
       patch                                                                   

Exception exceptions.AttributeError '_shutdown' in <module 'thre  718 days
       http://bugs.python.org/issue1947    nadiasvertex                  
                                                                               

Backport dictviews to 2.7                                         713 days
       http://bugs.python.org/issue1967    alexandre.vassalotti          
       patch, needs review                                                     

Backport set and dict comprehensions                              665 days
       http://bugs.python.org/issue2333    alexandre.vassalotti          
       patch, 26backport                                                       

Backport set literals                                             663 days
       http://bugs.python.org/issue2335    alexandre.vassalotti          
       patch, 26backport                                                       

PYTHON3PATH environment variable to supersede PYTHONPATH for mul    1 days
       http://bugs.python.org/issue2375    lemburg                       
                                                                               

Extension module build fails for MinGW: missing vcvarsall.bat     625 days
       http://bugs.python.org/issue2698    cmcqueen1975                  
                                                                               

arg 2 of PyErr_SetFromErrnoWithFilename should be const           619 days
       http://bugs.python.org/issue2758    brian.curtin                  
                                                                               

Trunk build issues in DragonFly BSD                               613 days
       http://bugs.python.org/issue2797    brian.curtin                  
       patch, needs review                                                     

Gzip cannot handle zero-padded output + patch                     610 days
       http://bugs.python.org/issue2846    pitrou                        
       patch, needs review                                                     

block operation on closed socket/pipe for multiprocessing         552 days
       http://bugs.python.org/issue3311    haypo                         
       patch, needs review                                                     

Add gamma function, error functions and other C99 math.h functio  543 days
       http://bugs.python.org/issue3366    mark.dickinson                
       patch, needs review                                                     

Macros for PyLong: sign, number of digits, fits in an int         425 days
       http://bugs.python.org/issue4294    mark.dickinson                
       patch                                                                   

reject unicode in zlib                                            379 days
       http://bugs.python.org/issue4757    haypo                         
       patch                                                                   

Distutils inappropriately reuses .o files between extension modu  320 days
       http://bugs.python.org/issue5372    tarek                         
       patch, patch, needs review                                              

Problem with email.MIME* library, using wrong new line            298 days
       http://bugs.python.org/issue5525    r.david.murray                
                                                                               

os.path.normpath doesn't preserve unicode                         263 days
       http://bugs.python.org/issue5827    ezio.melotti                  
       patch                                                                   

smtplib exception smtp.connect TypeError encode_plain             173 days
       http://bugs.python.org/issue6523    r.david.murray                
                                                                               

email.parser clips trailing \n of multipart/mixed part if part e  152 days
       http://bugs.python.org/issue6681    r.david.murray                
       patch                                                                   

Optimize PyBytes_FromObject.                                      151 days
       http://bugs.python.org/issue6688    alexandre.vassalotti          
       patch                                                                   

in xmlrpclib.py: NameError: global name 'HTTPSConnection' is not  143 days
       http://bugs.python.org/issue6769    krisvale                      
                                                                               

UnicodeDecodeError when retrieving binary data from cgi.FieldSto  126 days
       http://bugs.python.org/issue6854    r.david.murray                
                                                                               

test_distutils failure                                              0 days
       http://bugs.python.org/issue6961    flox                          
       buildbot                                                                

tmpnam should not be used if tempfile or mkstemp are available    110 days
       http://bugs.python.org/issue6965    flox                          
                                                                               

test_urllib: unsetting missing 'env' variable                       0 days
       http://bugs.python.org/issue7026    orsenthil                     
                                                                               

distutils and IronPython compatibility                             73 days
       http://bugs.python.org/issue7071    tarek                         
       26backport                                                              

weak dict iterators are fragile because of unpredictable GC runs   89 days
       http://bugs.python.org/issue7105    pitrou                        
       patch                                                                   

email:  msg.items() returns different values before and after ms   89 days
       http://bugs.python.org/issue7119    r.david.murray                
       patch                                                                   

get_payload(decode=True) eats last newline                         87 days
       http://bugs.python.org/issue7143    r.david.murray                
                                                                               

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo   49 days
       http://bugs.python.org/issue7382    alexandre.vassalotti          
       patch, easy                                                             

Document inspect.get(full)argspec limitation to Python function    39 days
       http://bugs.python.org/issue7422    georg.brandl                  
                                                                               

Py3.1: Fatal Python Error: Py_Initialize...unknown encoding: chc   35 days
       http://bugs.python.org/issue7441    georg.brandl                  
                                                                               

when piping output between subprocesses some fd is left open blo   36 days
       http://bugs.python.org/issue7448    flox                          
                                                                               

Solaris SPARC: _multiprocessing.so: symbol sem_timedwait: refere   35 days
       http://bugs.python.org/issue7454    srid                          
                                                                               

cPickle: stack underflow in load_pop()                             33 days
       http://bugs.python.org/issue7455    haypo                         
       patch                                                                   

crash in str.rfind() with an invalid start value                   33 days
       http://bugs.python.org/issue7458    haypo                         
       patch                                                                   

test_multiprocessing test_rapid_restart fails if port 9999 alrea   27 days
       http://bugs.python.org/issue7498    r.david.murray                
       patch, buildbot                                                         

Extended slicing with classic class behaves strangely               3 days
       http://bugs.python.org/issue7532    mark.dickinson                
       patch                                                                   

stringlib fastsearch could be improved on 64-bit builds            13 days
       http://bugs.python.org/issue7607    pitrou                        
                                                                               

distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option    7 days
       http://bugs.python.org/issue7617    tarek                         
       patch                                                                   

[patch] improve unicode methods: split() rsplit() and replace()    10 days
       http://bugs.python.org/issue7622    pitrou                        
       patch                                                                   

bytearray needs more tests for "b.some_method()[0] is not b"       10 days
       http://bugs.python.org/issue7625    pitrou                        
       patch                                                                   

test_urllib2 fails on Windows if not run from C:                    4 days
       http://bugs.python.org/issue7648    orsenthil                     
       easy                                                                    

[patch] Enable additional bytes and memoryview tests.               5 days
       http://bugs.python.org/issue7654    pitrou                        
       patch                                                                   

test_ctypes failure on AIX 5.3                                      4 days
       http://bugs.python.org/issue7657    mallayya                      
       patch                                                                   

Two float('nan') are not equal                                      0 days
       http://bugs.python.org/issue7660    mark.dickinson                
                                                                               

UCS4 build incorrectly translates cases for non-BMP code points     1 days
       http://bugs.python.org/issue7663    lemburg                       
                                                                               

python logger does not handle IOError Exception                     1 days
       http://bugs.python.org/issue7664    vinay.sajip                   
                                                                               

test_lib2to3 fails if path contains space                           0 days
       http://bugs.python.org/issue7666    benjamin.peterson             
                                                                               

test_unicode_file fails with non-ascii path                         2 days
       http://bugs.python.org/issue7669    ezio.melotti                  
                                                                               

Incorrect division in [wave.py]                                     1 days
       http://bugs.python.org/issue7681    benjamin.peterson             
       patch, needs review                                                     

Wrong link in HTML documentation                                    0 days
       http://bugs.python.org/issue7683    ezio.melotti                  
                                                                               

minor typo in re docs                                               0 days
       http://bugs.python.org/issue7685    ezio.melotti                  
       patch                                                                   

Wrong PEP number in importlib module manual page                    0 days
       http://bugs.python.org/issue7690    brett.cannon                  
                                                                               

test_bsddb3 files are not always removed when test fails            2 days
       http://bugs.python.org/issue7691    flox                          
       buildbot                                                                

Pointless comparision of unsigned integer with zero                 0 days
       http://bugs.python.org/issue7692    mark.dickinson                
                                                                               

os.getcwd() should raise UnicodeDecodeError for arbitrary bytes     0 days
       http://bugs.python.org/issue7697    pjenvey                       
                                                                               

pystack macro in Misc/gdbinit incorrectly uses PyEval_EvalFrame     0 days
       http://bugs.python.org/issue7698    skip.montanaro                
       patch                                                                   

"-3" flag does not work anymore                                     1 days
       http://bugs.python.org/issue7700    brett.cannon                  
       patch                                                                   

fix output string length for binascii.b2a_uu()                      1 days
       http://bugs.python.org/issue7701    pitrou                        
       patch                                                                   

Math calculation problem (1.6-1.0)>0.6, python said TRUE            0 days
       http://bugs.python.org/issue7704    tim_one                       
                                                                               

Support for sending multipart form data                          2452 days
       http://bugs.python.org/issue727898  r.david.murray                
                                                                               

email.MIME*.as_string removes duplicate spaces                   1440 days
       http://bugs.python.org/issue1422094 r.david.murray                
       easy                                                                    

test_parsedate_acceptable_to_time_functions+DST == :-(           1394 days
       http://bugs.python.org/issue1454285 r.david.murray                
                                                                               

email module does not complay with RFC 2046: CRLF issue          1196 days
       http://bugs.python.org/issue1571841 r.david.murray                
                                                                               

email.FeedParser.BufferedSubFile improperly handles "\r\n"        969 days
       http://bugs.python.org/issue1721862 r.david.murray                
       patch, easy                                                             



Top Issues Most Discussed (10)
______________________________

 20 dtoa.c: oversize b in quorem                                      11 days
open    http://bugs.python.org/issue7632   

 18 _ssl module causes segfault                                        5 days
open    http://bugs.python.org/issue7672   

 12 Speed up cPickle's pickling generally                            287 days
open    http://bugs.python.org/issue5683   

  8 OS X pythonw.c compile error with 10.4 or earlier deployment ta    7 days
open    http://bugs.python.org/issue7658   

  8 Fatal error on thread creation in low memory condition            27 days
open    http://bugs.python.org/issue7544   

  8 Direct calls to PyObject_Del/PyObject_DEL are broken for --with  558 days
open    http://bugs.python.org/issue3299   

  7 "-3" flag does not work anymore                                    1 days
closed  http://bugs.python.org/issue7700   

  7 tarfile.extractall can't have unicode extraction path              2 days
open    http://bugs.python.org/issue7693   

  7 test_urllib: unsetting missing 'env' variable                      0 days
closed  http://bugs.python.org/issue7026   

  7 os.path.abspath with unicode argument should call os.getcwdu     542 days
open    http://bugs.python.org/issue3426   




From g.brandl at gmx.net  Sat Jan 16 21:05:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 16 Jan 2010 21:05:46 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<20100113200840.GC14858@phd.pp.ru>
	<319e029f1001131227jf4006d3ya562fee26d22f3dd@mail.gmail.com>
Message-ID: <hit67o$7v4$1@ger.gmane.org>

Am 13.01.2010 21:27, schrieb Lennart Regebro:
> On Wed, Jan 13, 2010 at 21:08, Oleg Broytman <phd at phd.pp.ru> wrote:
>> On Wed, Jan 13, 2010 at 08:50:59PM +0100, Lennart Regebro wrote:
>>> What do you need to do in the PYTHONSTARTUP file?
>>> Ten years of Python programming, and I didn't even know it existed. :-)
>>
>>   See http://phd.pp.ru/Software/dotfiles/init.py.html for an example.
> 
> Cheese and fries! :-)
> 
> Anyway, I don't see how this could pose any problems, because any user
> advanced enough to do these things will be advanced enough to
> understand what the problem is and fix it so it works under Python 3
> too.

I'd propose adding a bit of text to the environment variables documentation,
and especially about the needs of PYTHONSTARTUP files.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From asmodai at in-nomine.org  Sat Jan 16 21:50:56 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Sat, 16 Jan 2010 21:50:56 +0100
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <874ompg225.fsf@brainbot.com>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
Message-ID: <20100116205056.GL99670@nexus.in-nomine.org>

-On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>hehe. tab completion:

With bpython and ipython available, why would you even want to stick to the
'plain old' interactive interpreter?

(Sorry to derail the discussion, but maybe there's more people that have not
heard of either or both.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
For ever, brother, hail and farewell...


From solipsis at pitrou.net  Sat Jan 16 21:57:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 16 Jan 2010 20:57:13 +0000 (UTC)
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <loom.20100116T215639-769@post.gmane.org>

Jeroen Ruigrok van der Werven <asmodai <at> in-nomine.org> writes:
> 
> -On [20100113 22:13], Ralf Schmitt (ralf <at> brainbot.com) wrote:
> >hehe. tab completion:
> 
> With bpython and ipython available, why would you even want to stick to the
> 'plain old' interactive interpreter?

Why wouldn't we?
There are probably many more people using the standard Python prompt than
ipython.





From ben+python at benfinney.id.au  Sat Jan 16 23:41:03 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2010 09:41:03 +1100
Subject: [Python-Dev] PYTHON3PATH
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>
	<87r5ptdhu3.fsf@muni.brainbot.com>
	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>
	<874ompg225.fsf@brainbot.com>
	<20100116205056.GL99670@nexus.in-nomine.org>
Message-ID: <87y6jxk70g.fsf@benfinney.id.au>

Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:

> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
> >hehe. tab completion:
>
> With bpython and ipython available, why would you even want to stick
> to the 'plain old' interactive interpreter?

Because those optional extras are not always available, whereas the
standard interactive interpreter is always available with CPython
distributions.

-- 
 \        ?I fly Air Bizarre. You buy a combination one-way round-trip |
  `\    ticket. Leave any Monday, and they bring you back the previous |
_o__)     Friday. That way you still have the weekend.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Sun Jan 17 01:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:22:10 +1000
Subject: [Python-Dev] PYTHON3PATH
In-Reply-To: <87y6jxk70g.fsf@benfinney.id.au>
References: <20100113172442.4A7771FA679@kimball.webabinitio.net>	<87r5ptdhu3.fsf@muni.brainbot.com>	<319e029f1001131150o7143f9bhacc9eb256ca30f76@mail.gmail.com>	<874ompg225.fsf@brainbot.com>	<20100116205056.GL99670@nexus.in-nomine.org>
	<87y6jxk70g.fsf@benfinney.id.au>
Message-ID: <4B525832.8090904@gmail.com>

Ben Finney wrote:
> Jeroen Ruigrok van der Werven <asmodai at in-nomine.org> writes:
> 
>> -On [20100113 22:13], Ralf Schmitt (ralf at brainbot.com) wrote:
>>> hehe. tab completion:
>> With bpython and ipython available, why would you even want to stick
>> to the 'plain old' interactive interpreter?
> 
> Because those optional extras are not always available, whereas the
> standard interactive interpreter is always available with CPython
> distributions.

I've never even contemplated the idea of installing 3rd party apps for
the 4 current development builds (2.x trunk, 2.x maintenance, 3.x trunk,
3.x maintenance) on my home development machine.

And of course work suffers from the problem of not being allowed to
install arbitrary apps I downloaded from the internet without getting
the licensing vetted for commercial acceptability.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 01:34:08 2010
From: nad at acm.org (Ned Deily)
Date: Sat, 16 Jan 2010 16:34:08 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
Message-ID: <nad-79FC16.16340816012010@news.gmane.org>

I've recently seen a couple of references to 3.1.2 go by in checkins 
which made me wonder whether dates have been proposed yet for updates to 
either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
references in the PEPs.  Some advance warning would be nice.  I assume 
that some critical problem could trigger planning for an update on short 
notice; is there a time-limit trigger as well?

-- 
 Ned Deily,
 nad at acm.org



From ncoghlan at gmail.com  Sun Jan 17 01:55:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 10:55:38 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52600A.7060201@gmail.com>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Usually there is one more regular maintenance release of the existing
maintenance branches within a few months of the creation of the next
version (releases before then are at the discretion of the release
manager, and security releases continue for much longer).

So take the planned 2.7 and 3.2 release dates and add a couple of months
to each one to get the likely release dates for the 2.6.x and 3.1.x
maintenance releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From martin at v.loewis.de  Sun Jan 17 10:21:49 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 17 Jan 2010 10:21:49 +0100
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <nad-79FC16.16340816012010@news.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <4B52D6AD.6090302@v.loewis.de>

Ned Deily wrote:
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.  I assume 
> that some critical problem could trigger planning for an update on short 
> notice; is there a time-limit trigger as well?

Barry was once proposing time-based releases; this idea didn't really
catch on.

It's the release manager who decides when the next bug fix release will
be made, often in response to developers asking for one.

Regards,
Martin



From solipsis at pitrou.net  Sun Jan 17 14:00:04 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 13:00:04 +0000 (UTC)
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
Message-ID: <loom.20100117T135910-313@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:
> 
> I've recently seen a couple of references to 3.1.2 go by in checkins 
> which made me wonder whether dates have been proposed yet for updates to 
> either 3.1 or 2.6.  I don't recall seeing any and I didn't see any 
> references in the PEPs.  Some advance warning would be nice.

There are a couple of release blockers right now. Once they are fixed or
deferred, I think it would be nice to have a 3.1.2.
Why do you need "some advance warning" though?




From ncoghlan at gmail.com  Sun Jan 17 14:40:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 17 Jan 2010 23:40:42 +1000
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <loom.20100117T135910-313@post.gmane.org>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
Message-ID: <4B53135A.7060104@gmail.com>

Antoine Pitrou wrote:
> Ned Deily <nad <at> acm.org> writes:
>> I've recently seen a couple of references to 3.1.2 go by in
>> checkins which made me wonder whether dates have been proposed yet
>> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
>> didn't see any references in the PEPs.  Some advance warning would
>> be nice.
> 
> There are a couple of release blockers right now. Once they are fixed
> or deferred, I think it would be nice to have a 3.1.2. Why do you
> need "some advance warning" though?

Advance warning does allow interested users that would consider
upgrading to schedule time for testing before the maintenance release
comes out. This is particularly useful in helping to make a 1-week RC
period effective in picking up issues that might otherwise lead to a
brown paper bag release to fix major issues that slipped through our own
automated test coverage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From nad at acm.org  Sun Jan 17 19:01:37 2010
From: nad at acm.org (Ned Deily)
Date: Sun, 17 Jan 2010 10:01:37 -0800
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
References: <nad-79FC16.16340816012010@news.gmane.org>
	<loom.20100117T135910-313@post.gmane.org>
	<4B53135A.7060104@gmail.com>
Message-ID: <nad-25165C.10013717012010@ger.gmane.org>

In article <4B53135A.7060104 at gmail.com>,
 Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine Pitrou wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> I've recently seen a couple of references to 3.1.2 go by in
> >> checkins which made me wonder whether dates have been proposed yet
> >> for updates to either 3.1 or 2.6.  I don't recall seeing any and I
> >> didn't see any references in the PEPs.  Some advance warning would
> >> be nice.
> > There are a couple of release blockers right now. Once they are fixed
> > or deferred, I think it would be nice to have a 3.1.2. Why do you
> > need "some advance warning" though?
> 
> Advance warning does allow interested users that would consider
> upgrading to schedule time for testing before the maintenance release
> comes out. This is particularly useful in helping to make a 1-week RC
> period effective in picking up issues that might otherwise lead to a
> brown paper bag release to fix major issues that slipped through our own
> automated test coverage.

That. and resource contention: there are always potential fixes in the 
pipeline that could or should be bumped in priority if one knows there 
is a code cutoff approaching.

-- 
 Ned Deily,
 nad at acm.org



From ziade.tarek at gmail.com  Sun Jan 17 20:51:58 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 20:51:58 +0100
Subject: [Python-Dev] Enhancing the shutil module
Message-ID: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>

Hello,

For 2.7/3.2, I am in the process of removing modules in Distutils that
can be replaced by calls to existing functions in stdlib. For
instance, "dir_util" and "file_util" (old modules from the Python 1.x
era) are going away in favor of calls to shutil (and os), so the
Distutils package gets lighter.

Another module I would like to move away from Distutils is
"archive_util". It contains helpers to build archives, whether they
are zip or tar files. I propose to move those useful functions into
shutil, as this seems the most logical place.

I also propose to maintain this "shutil" module for now on (no one is
declared as a maintainer in maintainers.rst) since Distutils will
become a heavy user of its functions.

Any objections/opinions ?

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 20:53:03 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 19:53:03 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <4B536A9F.5050203@voidspace.org.uk>

On 17/01/2010 19:51, Tarek Ziad? wrote:
> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>
> Regards,
> Tarek
>
>    
I think it's a great idea. :-)

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From brett at python.org  Sun Jan 17 20:55:47 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 17 Jan 2010 11:55:47 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>

On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>

If it's archive-agnostic then shutil is probably the best place.


>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
>
Great!


> Any objections/opinions ?
>

No objections from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100117/f97ac6eb/attachment-0007.htm>

From solipsis at pitrou.net  Sun Jan 17 21:04:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Jan 2010 20:04:41 +0000 (UTC)
Subject: [Python-Dev] Enhancing the shutil module
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <loom.20100117T210307-719@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.

Are these functions portable? Do they rely on external programs?

> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.

It's ok for me.

Regards

Antoine.




From ziade.tarek at gmail.com  Sun Jan 17 21:09:18 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:09:18 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
Message-ID: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>

On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, Jan 17, 2010 at 11:51, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Hello,
>>
>> For 2.7/3.2, I am in the process of removing modules in Distutils that
>> can be replaced by calls to existing functions in stdlib. For
>> instance, "dir_util" and "file_util" (old modules from the Python 1.x
>> era) are going away in favor of calls to shutil (and os), so the
>> Distutils package gets lighter.
>>
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>
> If it's archive-agnostic then shutil is probably the best place.

In more details:
It allows the creation of gzip, bzip2, tar and zip files through a single API.
There's a registry of supported formats and the API is driven by a
format identifier.

To do the work it uses stdlib's compression modules. Although it tries
the "zip" system command as a fallback if the "zipfile" module is not
present.

(notice that I've removed the support of "compress" (.Z) some time ago)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From fuzzyman at voidspace.org.uk  Sun Jan 17 21:15:00 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 17 Jan 2010 20:15:00 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <loom.20100117T210307-719@post.gmane.org>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
Message-ID: <4B536FC4.9090304@voidspace.org.uk>

On 17/01/2010 20:04, Antoine Pitrou wrote:
> Tarek Ziad?<ziade.tarek<at>  gmail.com>  writes:
>    
>> Another module I would like to move away from Distutils is
>> "archive_util". It contains helpers to build archives, whether they
>> are zip or tar files. I propose to move those useful functions into
>> shutil, as this seems the most logical place.
>>      
> Are these functions portable? Do they rely on external programs?
>
>    

I believe that part of the work that Tarek has been doing has been to 
make these distutils commands use the Python standard library and not 
depend on external programs. In which case they seem like *excellent* 
additions to the shutil module.

Of course Tarek can speak for himself...

Michael

>> I also propose to maintain this "shutil" module for now on (no one is
>> declared as a maintainer in maintainers.rst) since Distutils will
>> become a heavy user of its functions.
>>      
> It's ok for me.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Sun Jan 17 21:43:05 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 21:43:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B536FC4.9090304@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
Message-ID: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>

On Sun, Jan 17, 2010 at 9:15 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>> Are these functions portable? Do they rely on external programs?
>>
>>
>
> I believe that part of the work that Tarek has been doing has been to make
> these distutils commands use the Python standard library and not depend on
> external programs. In which case they seem like *excellent* additions to the
> shutil module.

yes, in the past the "tar" files where created using the "tar" command
but this has been changed. For a while now, they are portable and they
use stdlib code only. A recent addition is to be able to define
user/group permissions in the tar files, thanks to Lars' work in the
tarfile module.

There's one remaining external call for "zip" done if the zip module
is not found, but I am happy to remove it and throw an exception if
it's not found, and keep the external "zip" call on Distutils side, so
shutil stays 100% stdlib-powered.

> Of course Tarek can speak for himself...

Thanks for explaining it ! :)

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From sridharr at activestate.com  Sun Jan 17 22:50:52 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Sun, 17 Jan 2010 13:50:52 -0800
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
Message-ID: <4B53863C.5060304@activestate.com>

On 1/17/2010 12:09 PM, Tarek Ziad? wrote:
> On Sun, Jan 17, 2010 at 8:55 PM, Brett Cannon<brett at python.org>  wrote:
>> >  On Sun, Jan 17, 2010 at 11:51, Tarek Ziad?<ziade.tarek at gmail.com>  wrote:
>>> >>  Another module I would like to move away from Distutils is
>>> >>  "archive_util". It contains helpers to build archives, whether they
>>> >>  are zip or tar files. I propose to move those useful functions into
>>> >>  shutil, as this seems the most logical place.
>> >  If it's archive-agnostic then shutil is probably the best place.
> In more details:
> It allows the creation of gzip, bzip2, tar and zip files through a single API.
> There's a registry of supported formats and the API is driven by a
> format identifier.

Will it also allow decompression of the said archive types? Distribute 
has some utility code to handle zip/tar archives. So does PyPM. This is 
because the `tarfile` and `zipfile` modules do not "just work" due to 
several issues.

See http://gist.github.com/279606

Take note of the following in the above code:

  1) _ensure_read_write_access
  2) *File.is_valid
  3) ZippedFile.extract ... issue 6510
  4) ZippedFile.extract ... issue 6609
  5) TarredFile.extract ... issue 6584
  6) The way unpack() detects the unpacked directory.

-srid


From ziade.tarek at gmail.com  Sun Jan 17 23:09:29 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 17 Jan 2010 23:09:29 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B53863C.5060304@activestate.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<bbaeab101001171155j1766b8ack591aab4ce2681c03@mail.gmail.com>
	<94bdd2611001171209l4a841dd2ne0848b9501d76470@mail.gmail.com>
	<4B53863C.5060304@activestate.com>
Message-ID: <94bdd2611001171409j4ec1e0bfj47e59894fdec0d8a@mail.gmail.com>

On Sun, Jan 17, 2010 at 10:50 PM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
[..]
> Will it also allow decompression of the said archive types?

No but it would make sense having this one as well.
Distribute/Setuptools' "unpack_archive" (used by easy_install) was
implemented using the same principle as Distutils' "make_archive".

I can add it as well while I am at it : it has been successfully used
for years by easy_install.

> Distribute has
> some utility code to handle zip/tar archives. So does PyPM. This is because
> the `tarfile` and `zipfile` modules do not "just work" due to several
> issues.
>
> See http://gist.github.com/279606
>
> Take note of the following in the above code:
>
> ?1) _ensure_read_write_access
> ?2) *File.is_valid
> ?3) ZippedFile.extract ... issue 6510
> ?4) ZippedFile.extract ... issue 6609
> ?5) TarredFile.extract ... issue 6584

Looks like some of these are already fixed (I just looked quickly in
the bug tracker though)
If its not already done and pending, it would be great if you could
refactor your fixes into patches for the remaining issues for each one
of those modules

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org


From barry at python.org  Sun Jan 17 23:56:37 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Jan 2010 17:56:37 -0500
Subject: [Python-Dev] 3.1.2 / 2.6.5 maintenance releases?
In-Reply-To: <4B52D6AD.6090302@v.loewis.de>
References: <nad-79FC16.16340816012010@news.gmane.org>
	<4B52D6AD.6090302@v.loewis.de>
Message-ID: <BEC881FD-2BDE-4B5C-A570-B8C1E23D6DE1@python.org>

On Jan 17, 2010, at 4:21 AM, Martin v. L?wis wrote:

> Barry was once proposing time-based releases; this idea didn't really
> catch on.

I'm still a proponent of this, but it doesn't seem like there's enough support for it.

> It's the release manager who decides when the next bug fix release will
> be made, often in response to developers asking for one.

I'm happy to make a 2.6.5 release whenever it makes sense.
-Barry



From ncoghlan at gmail.com  Mon Jan 18 13:40:01 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Jan 2010 22:40:01 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
Message-ID: <4B5456A1.2080109@gmail.com>

Tarek Ziad? wrote:
> There's one remaining external call for "zip" done if the zip module
> is not found, but I am happy to remove it and throw an exception if
> it's not found, and keep the external "zip" call on Distutils side, so
> shutil stays 100% stdlib-powered.

+1 for that approach. These changes all sound like nice additions to
shutil, and hooray for every module that gets adopted by an active
maintainer :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Mon Jan 18 14:34:14 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 14:34:14 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5456A1.2080109@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
Message-ID: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>

On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
> 
> Tarek Ziad? wrote:
>> There's one remaining external call for "zip" done if the zip module
>> is not found, but I am happy to remove it and throw an exception if
>> it's not found, and keep the external "zip" call on Distutils side, so
>> shutil stays 100% stdlib-powered.
> 
> +1 for that approach. These changes all sound like nice additions to
> shutil, and hooray for every module that gets adopted by an active
> maintainer :)

Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).

Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

From doug.hellmann at gmail.com  Mon Jan 18 14:46:05 2010
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 18 Jan 2010 08:46:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils  
>>> side, so
>>> shutil stays 100% stdlib-powered.
>>
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
>
> Isn't it a bit weird to include that to shutil though? shutil  
> advertises itself as "a number of high-level operations on files and  
> collections of files." and from what I understood it was a bunch of  
> shell-type utility functions to easily copy, move or remove files  
> and directories (that's pretty much all there is in it at this time).
>
> Wouldn't it make more sense to put those "archive utils" functions/ 
> objects in a new module separate from shutil, dealing specifically  
> with cross-archive APIs and linked from the current archive-specific  
> modules (essentially, just take the current archive_util, move it to  
> the toplevel of the stdlib and maybe rename it)? It would also make  
> the module much easier to find when searching through the module  
> listing, I think.

+1



From fuzzyman at voidspace.org.uk  Mon Jan 18 14:57:49 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 18 Jan 2010 13:57:49 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
Message-ID: <4B5468DD.5040200@voidspace.org.uk>

On 18/01/2010 13:46, Doug Hellmann wrote:
>
> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>
>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>
>>> Tarek Ziad? wrote:
>>>> There's one remaining external call for "zip" done if the zip module
>>>> is not found, but I am happy to remove it and throw an exception if
>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>> shutil stays 100% stdlib-powered.
>>>
>>> +1 for that approach. These changes all sound like nice additions to
>>> shutil, and hooray for every module that gets adopted by an active
>>> maintainer :)
>>
>> Isn't it a bit weird to include that to shutil though? shutil 
>> advertises itself as "a number of high-level operations on files and 
>> collections of files." 

Well - isn't what's being proposed "a number of high-level operations on 
files and collections of files." ?

>> and from what I understood it was a bunch of shell-type utility functions

like tar and zip? :-)

>> to easily copy, move or remove files and directories (that's pretty 
>> much all there is in it at this time).
>>

I don't think the additions are out of place prima-facie.

>> Wouldn't it make more sense to put those "archive utils" 
>> functions/objects in a new module separate from shutil, dealing 
>> specifically with cross-archive APIs and linked from the current 
>> archive-specific modules (essentially, just take the current 
>> archive_util, move it to the toplevel of the stdlib and maybe rename 
>> it)? It would also make the module much easier to find when searching 
>> through the module listing, I think.
>
> +1
>

Proliferation of modules is itself a bad thing though.

Michael


> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.




From ziade.tarek at gmail.com  Mon Jan 18 15:34:12 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 15:34:12 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <94bdd2611001180634sacd998fm6f67dc680e2e61c3@mail.gmail.com>

On Mon, Jan 18, 2010 at 2:57 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
[..]
>>> Wouldn't it make more sense to put those "archive utils"
>>> functions/objects in a new module separate from shutil, dealing specifically
>>> with cross-archive APIs and linked from the current archive-specific modules
>>> (essentially, just take the current archive_util, move it to the toplevel of
>>> the stdlib and maybe rename it)? It would also make the module much easier
>>> to find when searching through the module listing, I think.
>>
>> +1
>>
>
> Proliferation of modules is itself a bad thing though.

I am with Michael here. I think having this function in shutil is the
right place.

For the find problem, I think docs.python.org is easy to search now,
as long as the shutil module documentation has an good set of examples
for this new API.

We could even add references in the tarfile, zipfile modules
documentation pointing to these examples.

Regards
Tarek
-- 
Tarek Ziad? | http://ziade.org


From masklinn at masklinn.net  Mon Jan 18 15:56:05 2010
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 18 Jan 2010 15:56:05 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <4B5468DD.5040200@voidspace.org.uk>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
Message-ID: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>

On 18 Jan 2010, at 14:57 , Michael Foord wrote:
> 
> On 18/01/2010 13:46, Doug Hellmann wrote:
>> 
>> On Jan 18, 2010, at 8:34 AM, Masklinn wrote:
>> 
>>> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>>>> 
>>>> Tarek Ziad? wrote:
>>>>> There's one remaining external call for "zip" done if the zip module
>>>>> is not found, but I am happy to remove it and throw an exception if
>>>>> it's not found, and keep the external "zip" call on Distutils side, so
>>>>> shutil stays 100% stdlib-powered.
>>>> 
>>>> +1 for that approach. These changes all sound like nice additions to
>>>> shutil, and hooray for every module that gets adopted by an active
>>>> maintainer :)
>>> 
>>> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." 
> 
> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
> 
Well no those are high-level operations on a very restricted set of file types (archives).

>>> and from what I understood it was a bunch of shell-type utility functions
> 
> like tar and zip? :-)
> 
Tar and zip have a module each at this time, so they're considered pretty big. I don't think anybody would consider "mv" big enough to give it a module on its own.

>>> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.
>> 
>> +1
>> 
> 
> Proliferation of modules is itself a bad thing though.
Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.

Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

From phd at phd.pp.ru  Mon Jan 18 16:03:38 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 18 Jan 2010 18:03:38 +0300
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <20100118150337.GA19391@phd.pp.ru>

On Mon, Jan 18, 2010 at 02:34:14PM +0100, Masklinn wrote:
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

   +1

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From ziade.tarek at gmail.com  Mon Jan 18 16:24:37 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 18 Jan 2010 16:24:37 +0100
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<2D626E52-E592-44C8-A8E8-C72FABE6AFEA@gmail.com>
	<4B5468DD.5040200@voidspace.org.uk>
	<6A054C33-75FB-48E6-9AD1-FE6F0ADF5940@masklinn.net>
Message-ID: <94bdd2611001180724u7f48e620ub32e13ef96c873b9@mail.gmail.com>

On Mon, Jan 18, 2010 at 3:56 PM, Masklinn <masklinn at masklinn.net> wrote:
[..]
>> Well - isn't what's being proposed "a number of high-level operations on files and collections of files." ?
>>
> Well no those are high-level operations on a very restricted set of file types (archives)

not really: make_archive/unpack_archive, are also dealing with files
and directories.

[..]
>> Proliferation of modules is itself a bad thing though.
> Yes, but so is the loss of focus of a module. I hate using that argument, but I fear this would be a step on a slippery slope of making shutil a monster-bag of anything that has to do with inodes, no matter how tenuous or removed the connection is.
>
> Plus having this as a toplevel module/package would open the window to moving all archive-related modules within it (in the py4 window), ? la xml package without having to move itself.

I am not sure why this would happen. For instance, shutil is already
on the top of the os module since a few major versions IIRC, because
it reads and writes files and directories. But it was not moved into
the os package (or vice-versa)

The shutil module uses APIs to read and write files. So if it works
with archives, it's just a specific read/write API that is used, but
that doesn't mean tarfile and zipfile might be reunited with shutil
imho.

If the shutil module is restricted to high-level files and directories
manipulation, working with archives is just a target like another I
think.

But at the end I am 0- to create a new module, because what really
matters to me is to take it out of Distutils :)

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org


From listsin at integrateddevcorp.com  Mon Jan 18 16:56:05 2010
From: listsin at integrateddevcorp.com (Steve Steiner (listsin))
Date: Mon, 18 Jan 2010 10:56:05 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
Message-ID: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>


On Jan 18, 2010, at 8:34 AM, Masklinn wrote:

> On 18 Jan 2010, at 13:40 , Nick Coghlan wrote:
>> 
>> Tarek Ziad? wrote:
>>> There's one remaining external call for "zip" done if the zip module
>>> is not found, but I am happy to remove it and throw an exception if
>>> it's not found, and keep the external "zip" call on Distutils side, so
>>> shutil stays 100% stdlib-powered.
>> 
>> +1 for that approach. These changes all sound like nice additions to
>> shutil, and hooray for every module that gets adopted by an active
>> maintainer :)
> 
> Isn't it a bit weird to include that to shutil though? shutil advertises itself as "a number of high-level operations on files and collections of files." and from what I understood it was a bunch of shell-type utility functions to easily copy, move or remove files and directories (that's pretty much all there is in it at this time).
> 
> Wouldn't it make more sense to put those "archive utils" functions/objects in a new module separate from shutil, dealing specifically with cross-archive APIs and linked from the current archive-specific modules (essentially, just take the current archive_util, move it to the toplevel of the stdlib and maybe rename it)? It would also make the module much easier to find when searching through the module listing, I think.

As much of a pain as it is to get new modules accepted, I agree that mixing archiving functions into shutil is not the right way to do it and that a separate archive_util module would make much more sense and would give a logical place to put any extensions to archive handling.

S





From rdmurray at bitdance.com  Mon Jan 18 20:28:47 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 18 Jan 2010 14:28:47 -0500
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
Message-ID: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>

On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
> As much of a pain as it is to get new modules accepted, I agree that
> mixing archiving functions into shutil is not the right way to do it
> and that a separate archive_util module would make much more sense and
> would give a logical place to put any extensions to archive handling.

Looking at the source code and API for both shutil and archive_util, I
think that the archive_util methods fit into shutil.  shutil currently
wraps some standard library facilities with convenience functions
for operations you might otherwise perform at the shell command line using
OS facilities.  As far as I can tell, archive_util does the same, and
seems quite within the shutil mission of "high level file operations".

So +1 from me for putting these in shutil.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From p.f.moore at gmail.com  Mon Jan 18 20:48:43 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 18 Jan 2010 19:48:43 +0000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
	<loom.20100117T210307-719@post.gmane.org>
	<4B536FC4.9090304@voidspace.org.uk>
	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>
	<4B5456A1.2080109@gmail.com>
	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>
	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>
	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
Message-ID: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>

2010/1/18 R. David Murray <rdmurray at bitdance.com>:
> On Mon, 18 Jan 2010 10:56:05 -0500, "Steve Steiner (listsin)" <listsin at integrateddevcorp.com> wrote:
>> As much of a pain as it is to get new modules accepted, I agree that
>> mixing archiving functions into shutil is not the right way to do it
>> and that a separate archive_util module would make much more sense and
>> would give a logical place to put any extensions to archive handling.
>
> Looking at the source code and API for both shutil and archive_util, I
> think that the archive_util methods fit into shutil. ?shutil currently
> wraps some standard library facilities with convenience functions
> for operations you might otherwise perform at the shell command line using
> OS facilities. ?As far as I can tell, archive_util does the same, and
> seems quite within the shutil mission of "high level file operations".
>
> So +1 from me for putting these in shutil.

Conceptually, I'm happy with these going into shutil (and +1 on the
rest of Tarek's proposal, too!)

To my mind, shutil is a module for higher-level operations on files -
the sort of things you'd do in shell commands, like move a batch of
files around (mv), create a directory tree (mkdir -p). Tarring or
zipping up a batch of files fits nicely into that space.

Paul.


From david.lyon at preisshare.net  Tue Jan 19 02:53:43 2010
From: david.lyon at preisshare.net (David Lyon)
Date: Mon, 18 Jan 2010 20:53:43 -0500
Subject: [Python-Dev] PyCon Keynote
In-Reply-To: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
References: <ca471dc21001131051i10f1b436nfb3dc71f1e2a25e2@mail.gmail.com>
Message-ID: <0dfb370163cf3a284ca256f49662db74@preisshare.net>

On Wed, 13 Jan 2010 10:51:14 -0800, Guido van Rossum <guido at python.org>
wrote:
> Please mail me topics you'd like to hear me talk about in my keynote
> at PyCon this year.

As a typical (but perphaps more vocal) python developer I would
like to hear things that might aspire me and others to move to 
python 3.

The way I see it is that python 2.x represents everyones 
comfort zone. It's safe and nobody got fired at work for 
using python 2.x

I would like to hear how python 3 could be 'shaken' up
with a slightly less conservative policy that has gone
with the python 2.x series.

The logic perhaps being that since only a minority
use the 3.x series, it's functionality set is still
more up in the air. imo it needs bigger batteries
to give it more power than the 2.x series.

This meaning that the stdlib could take an extra
5-10 packages not in the 2.x series. Just as
one example, sphinx - a great documentation
tool. I can easily name five or six others.

I'd also like to hear more of your ideas on pypi
and getting it to have as much who-ha as CPAN.

You could do a lot to enlarge the developer
group. Help us all get our priorities to be
inline with your own wishes and expectations.

If you ask us all to put in a big year and
buy you a beer at the end.. I'm certain we
all will.

Every little bit of inspiration and direction
counts for a lot...

Best Regards

David











From ncoghlan at gmail.com  Tue Jan 19 12:20:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Jan 2010 21:20:05 +1000
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>	<loom.20100117T210307-719@post.gmane.org>	<4B536FC4.9090304@voidspace.org.uk>	<94bdd2611001171243i604b612ct2db654ffe3812ec8@mail.gmail.com>	<4B5456A1.2080109@gmail.com>	<96E0938E-D1E1-4751-A06B-2B2BD13830BA@masklinn.net>	<E75AB1B5-7C9D-4C4A-ABF9-59B68A677AE7@integrateddevcorp.com>	<20100118192847.1C99C1FB6FE@kimball.webabinitio.net>
	<79990c6b1001181148y7e48d770n546acfe2f6c62367@mail.gmail.com>
Message-ID: <4B559565.8090602@gmail.com>

Paul Moore wrote:
> 2010/1/18 R. David Murray <rdmurray at bitdance.com>:
>> So +1 from me for putting these in shutil.
> 
> Conceptually, I'm happy with these going into shutil (and +1 on the
> rest of Tarek's proposal, too!)
> 
> To my mind, shutil is a module for higher-level operations on files -
> the sort of things you'd do in shell commands, like move a batch of
> files around (mv), create a directory tree (mkdir -p). Tarring or
> zipping up a batch of files fits nicely into that space.

This is also reflected in the way at least Windows handles archives
these days - it took them a couple of iterations to get it right (and
resolve some of the performance impacts), but Explorer now does a decent
job of integrating archives into the directory tree as "folders that
happen to be compressed".

Are archives as fundamental as directories and files? No. But in the
context of shutil, the fact that their internal structure is largely
about directories and files makes them more than just another arbitrary
file type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From barry at python.org  Tue Jan 19 14:16:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 08:16:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
Message-ID: <20100119081639.5d431ed9@freewill>

I've just updated the Launchpad mirrors for the 4 active Python branches,
trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar branches
on code.python.org but it's probably been 7 months or so since those were
regularly updated.  Now the Launchpad branches sync against the read-only
Subversion branches at http://svn.python.org, so they should remain up-to-date
(within the re-sync timeframe of about 4 hours).

This means you can once again use Bazaar to get local branches of Python, and
you can of course push your own branches to Launchpad.  I believe you can even
use the bzr-svn plugin to commit changes back to the Subversion master, though
I have not yet tried this.

To get a local branch, just do any of the following:

    % bzr branch lp:python (for trunk)
    % bzr branch lp:python/2.6
    % bzr branch lp:python/py3k
    % bzr branch lp:python/3.1

(It's fairly easy to create new mirrors for other Subversion branches,
e.g. Python 2.5; just drop me an email if you want them.)

If you're going to create a lot of branches you probably want to put them in a
shared repository.  E.g.

    % bzr init-repo pythonbzr
    % cd pythonbzr
    % bzr branch lp:python/py3k

Bazaar 2.0 or better is recommended.  For me, it took about 5m to check the
first branch out from Launchpad, and then about 30s or so for each subsequent
branch.

Enjoy,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/89a42d75/attachment-0005.pgp>

From abpillai at gmail.com  Tue Jan 19 14:37:18 2010
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Tue, 19 Jan 2010 19:07:18 +0530
Subject: [Python-Dev] Enhancing the shutil module
In-Reply-To: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
References: <94bdd2611001171151w21838b09k4620c562058d9ccc@mail.gmail.com>
Message-ID: <8548c5f31001190537l2bcab5cfkb6f461910e4abd8b@mail.gmail.com>

On Mon, Jan 18, 2010 at 1:21 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Hello,
>
> For 2.7/3.2, I am in the process of removing modules in Distutils that
> can be replaced by calls to existing functions in stdlib. For
> instance, "dir_util" and "file_util" (old modules from the Python 1.x
> era) are going away in favor of calls to shutil (and os), so the
> Distutils package gets lighter.
>
> Another module I would like to move away from Distutils is
> "archive_util". It contains helpers to build archives, whether they
> are zip or tar files. I propose to move those useful functions into
> shutil, as this seems the most logical place.
>
> I also propose to maintain this "shutil" module for now on (no one is
> declared as a maintainer in maintainers.rst) since Distutils will
> become a heavy user of its functions.
>
> Any objections/opinions ?
>

+1 for this. Just make sure that you change the docstring of shutil
 which now reads as,

" shutil - Utility functions for copying files and directory trees."

 According to this "definition", archives don't fit in there. But the
 functionality does fit right in, so just need to make sure that it
 is reflected in the __doc__ .


> Regards,
> Tarek
>
> --
> Tarek Ziad? | http://ziade.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/55892759/attachment-0005.htm>

From vinay_sajip at yahoo.co.uk  Tue Jan 19 16:50:42 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 19 Jan 2010 15:50:42 +0000 (UTC)
Subject: [Python-Dev] Mailing List archive corruption?
Message-ID: <loom.20100119T164757-651@post.gmane.org>

Hi,

When I look at the mailing list archive for python-dev, I see some odd stuff at
the bottom of the page:

http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232

Anyone know what's happened?

Regards,

Vinay Sajip



From barry at python.org  Tue Jan 19 17:07:48 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:07:48 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <loom.20100119T164757-651@post.gmane.org>
References: <loom.20100119T164757-651@post.gmane.org>
Message-ID: <20100119110748.69bc564a@freewill>

On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:

>When I look at the mailing list archive for python-dev, I see some odd stuff at
>the bottom of the page:
>
>http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>
>Anyone know what's happened?

WTF?  I think the archives were recently regenerated, so there's probably a
fubar there.  CC'ing the postmasters.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/8947a2e9/attachment-0005.pgp>

From foom at fuhm.net  Tue Jan 19 17:24:57 2010
From: foom at fuhm.net (James Y Knight)
Date: Tue, 19 Jan 2010 11:24:57 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119110748.69bc564a@freewill>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
Message-ID: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>


On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:

> On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
>
>> When I look at the mailing list archive for python-dev, I see some  
>> odd stuff at
>> the bottom of the page:
>>
>> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
>>
>> Anyone know what's happened?
>
> WTF?  I think the archives were recently regenerated, so there's  
> probably a
> fubar there.  CC'ing the postmasters.

That happens if messages had unescaped "From" lines in the middle of  
them.

No doubt, you've now broken every link anyone had ever made into the  
python-dev archives, because now all the article numbers are  
different. BTDT...unfortunately... Pipermail really is quite crappy,  
sigh.

Anyhow, when I did that, I went back to a backup to get the original  
article numbers, and edited the mbox file escaping From lines or  
adding additional empty messages until the newly regenerated article  
numbers matched the originals. I'd highly recommend going through that  
painful process, since I suspect a *lot* of people have links to the  
python-dev archive. Hope you have a backup (or can find caches on  
google or archive.org or something).

James


From barry at python.org  Tue Jan 19 17:44:21 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 11:44:21 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119114421.3b96fbd4@freewill>

On Jan 19, 2010, at 11:24 AM, James Y Knight wrote:

>No doubt, you've now broken every link anyone had ever made into the  
>python-dev archives, because now all the article numbers are  
>different. BTDT...unfortunately... Pipermail really is quite crappy,  
>sigh.

I've been trying for 10+ years to get folks interested in helping me fix this
(and a few other warts) in Pipermail, to not much success. ;/

>Anyhow, when I did that, I went back to a backup to get the original  
>article numbers, and edited the mbox file escaping From lines or  
>adding additional empty messages until the newly regenerated article  
>numbers matched the originals. I'd highly recommend going through that  
>painful process, since I suspect a *lot* of people have links to the  
>python-dev archive. Hope you have a backup (or can find caches on  
>google or archive.org or something).

bin/cleanarch uses a set of heuristics to find unescaped From lines and fix
them.  It's generally pretty good, but it certain can change message numbers
(and sadly, their urls).

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/e32aa882/attachment-0005.pgp>

From rdmurray at bitdance.com  Tue Jan 19 18:48:41 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Jan 2010 12:48:41 -0500
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
References: <loom.20100119T164757-651@post.gmane.org>
	<20100119110748.69bc564a@freewill>
	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
Message-ID: <20100119174841.9BC3B16A53@kimball.webabinitio.net>

On Tue, 19 Jan 2010 11:24:57 -0500, James Y Knight <foom at fuhm.net> wrote:
> 
> On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote:
> 
> > On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote:
> >
> >> When I look at the mailing list archive for python-dev, I see some  
> >> odd stuff at the bottom of the page:
> >>
> >> http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232
> >>
> >> Anyone know what's happened?
> >
> > WTF?  I think the archives were recently regenerated, so there's  
> > probably a fubar there.  CC'ing the postmasters.
> 
> That happens if messages had unescaped "From" lines in the middle of  
> them.
> 
> No doubt, you've now broken every link anyone had ever made into the  
> python-dev archives, because now all the article numbers are  
> different. BTDT...unfortunately... Pipermail really is quite crappy,  
> sigh.
> 
> Anyhow, when I did that, I went back to a backup to get the original  
> article numbers, and edited the mbox file escaping From lines or  
> adding additional empty messages until the newly regenerated article  
> numbers matched the originals. I'd highly recommend going through that  
> painful process, since I suspect a *lot* of people have links to the  
> python-dev archive. Hope you have a backup (or can find caches on  
> google or archive.org or something).

The Python issue tracker does, for one.

--
R. David Murray                                      www.bitdance.com
Business Process Automation - Network/Server Management - Routers/Firewalls


From ncoghlan at gmail.com  Tue Jan 19 22:18:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 07:18:52 +1000
Subject: [Python-Dev] Mailing List archive corruption?
In-Reply-To: <20100119174841.9BC3B16A53@kimball.webabinitio.net>
References: <loom.20100119T164757-651@post.gmane.org>	<20100119110748.69bc564a@freewill>	<BAC0620A-3D5F-4AAB-8A2E-6F0E3913A4EA@fuhm.net>
	<20100119174841.9BC3B16A53@kimball.webabinitio.net>
Message-ID: <4B5621BC.4070608@gmail.com>

R. David Murray wrote:
> The Python issue tracker does, for one.

And all the PEPs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From david.lyon at pythontest.org  Wed Jan 20 00:16:44 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 10:16:44 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119081639.5d431ed9@freewill>
References: <20100119081639.5d431ed9@freewill>
Message-ID: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>


Hi Barry,

That looks very interesting...

So does that mean we could update the stdlib for a given
python version using this ?

David

> I've just updated the Launchpad mirrors for the 4 active Python branches,
> trunk, py3k, 2.6, and 3.1.  These used to mirror the defunct Bazaar
> branches
> on code.python.org but it's probably been 7 months or so since those were
> regularly updated.  Now the Launchpad branches sync against the read-only
> Subversion branches at http://svn.python.org, so they should remain
> up-to-date
> (within the re-sync timeframe of about 4 hours).
>
> This means you can once again use Bazaar to get local branches of Python,
> and
> you can of course push your own branches to Launchpad.  I believe you can
> even
> use the bzr-svn plugin to commit changes back to the Subversion master,
> though
> I have not yet tried this.
>
> To get a local branch, just do any of the following:
>
>     % bzr branch lp:python (for trunk)
>     % bzr branch lp:python/2.6
>     % bzr branch lp:python/py3k
>     % bzr branch lp:python/3.1
>
> (It's fairly easy to create new mirrors for other Subversion branches,
> e.g. Python 2.5; just drop me an email if you want them.)
>
> If you're going to create a lot of branches you probably want to put them
> in a
> shared repository.  E.g.
>
>     % bzr init-repo pythonbzr
>     % cd pythonbzr
>     % bzr branch lp:python/py3k
>
> Bazaar 2.0 or better is recommended.  For me, it took about 5m to check
> the
> first branch out from Launchpad, and then about 30s or so for each
> subsequent
> branch.
>
> Enjoy,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/david.lyon%40pythontest.org
>




From barry at python.org  Wed Jan 20 00:54:39 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 18:54:39 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119185439.299660c7@freewill>

On Jan 20, 2010, at 10:16 AM, David Lyon wrote:

>Hi Barry,
>
>That looks very interesting...

Hi David,

>So does that mean we could update the stdlib for a given
>python version using this ?

In a sense, yes (if I understand your question correctly).

You can use Bazaar to branch any of the 4 Python series and use all the modern
DVCS goodness to develop your updates.  You can share your branches with
others via e.g. Launchpad and even request reviews (called "merge proposals")
to get feedback from others.

The one thing I am unsure about, mostly because I have not tried it, is
whether your Bazaar branch can be used to commit directly back to the Python
Subversion master branches.  I /think/ the answer is yes, assuming of course
that you have permission to do so, and that you have a modern version of
Bazaar and the bzr-svn plugin.

It might even be possible to commit your Bazaar branch to a local Subversion
branch, and then commit the latter to get it pushed up to svn.python.org.

At worst, you would use Bazaar's features to get your patch into a state you
and your reviewers are happy with, then you would generate a diff for
application to your copy of the svn.python.org branch.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/37753b1a/attachment-0005.pgp>

From david.lyon at pythontest.org  Wed Jan 20 01:51:23 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 11:51:23 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119185439.299660c7@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
Message-ID: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>

> On Jan 20, 2010, at 10:16 AM, Barry wrote:

>> So does that mean we could update the stdlib for a given
>> python version using this ?
>
> In a sense, yes (if I understand your question correctly).

Yeah, it just needs an implementation.

> The one thing I am unsure about, mostly because I have not tried it, is
> whether your Bazaar branch can be used to commit directly back to the
> Python Subversion master branches.  I /think/ the answer is yes,
> assuming of course that you have permission to do so...

Well I'm too Senior and my stuff is too forward looking to qualify
for that just yet.

I'd be happy to see bzr and mercurial and git all made it together
into the stdlib for python 3. That would give a superb updating
mechanism for python that would propel python well beyond
the dinosaur badlands of CPAN and other languages.

I was actually reading from
(http://en.wikipedia.org/wiki/Python_%28programming_language%29):

"Rather than requiring all desired functionality to be built into the
language's core, Python was designed to be highly extensible. .. .. This
design of a small core language with a large standard library and an
easily extensible interpreter was intended by Van Rossum from the very
start because of his frustrations with ABC (which espoused the opposite
mindset).[5]"

To me, the source code control systems seem to be fully in tune
with the original design of python. That is, to be able to
easily pull external libraries in.

I think what has changed is that the mechanisms now (the SCMs)
are way more highly developed than before. Apart from that
though, after reading the full wikipedia article I'm left
with the distinct impression that things are still pretty
much the same (in that python design philosophy is advanced),
just that the landscape (of external C libraries) has changed.

Now all the libraries are external (on the internet) and
all externally managed.

So with just a tiny amount of work, imho we could pull
it all together to bring python 3 *back* to being that
cool tool that it once was (not saying it isn't now).

Were you offering me an experimental branch somewhere
for python 3 SCM integration ?

David







From jnoller at gmail.com  Wed Jan 20 02:09:15 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 20:09:15 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>

On Tue, Jan 19, 2010 at 7:51 PM, David Lyon <david.lyon at pythontest.org> wrote:
>> On Jan 20, 2010, at 10:16 AM, Barry wrote:
>
>>> So does that mean we could update the stdlib for a given
>>> python version using this ?
>>
>> In a sense, yes (if I understand your question correctly).
>
> Yeah, it just needs an implementation.
>
>> The one thing I am unsure about, mostly because I have not tried it, is
>> whether your Bazaar branch can be used to commit directly back to the
>> Python Subversion master branches. ?I /think/ the answer is yes,
>> assuming of course that you have permission to do so...
>
> Well I'm too Senior and my stuff is too forward looking to qualify
> for that just yet.
>
> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

i sincerely doubt that a source control system will be included in the
standard library in the future. Especially 3. A SCM is not a "package
management system".

Barry was talking about mirrors of the python code. It is true a
"package manager" could be developed based on a SCM, however you need
to implement this far away from the stdlib and get traction with it
within the community long before inclusion would be considered.

The decision to move python's source control from SVN to mercurial was
controversial enough; including 3 or more scm libraries into core
would be an intractable uphill mountain of bike sheds.

> So with just a tiny amount of work, imho we could pull
> it all together to bring python 3 *back* to being that
> cool tool that it once was (not saying it isn't now).

Python 3 is still modularized, still has a standard library, etc. If
you're really interested in helping with the standard library, get on
stdlib-sig, and get ready to write code and PEPs.

> Were you offering me an experimental branch somewhere
> for python 3 SCM integration ?

Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

jesse


From barry at python.org  Wed Jan 20 04:32:30 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 22:32:30 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <20100119223230.4c4a7ed5@freewill>

On Jan 19, 2010, at 08:09 PM, Jesse Noller wrote:

>The decision to move python's source control from SVN to mercurial was
>controversial enough; including 3 or more scm libraries into core
>would be an intractable uphill mountain of bike sheds.

I'd be surprised if any of the big 3 DVCS developers would actually /want/
their stuff in the stdlib.  Being in the stdlib has its advantages and
disadvantages.  I think for rapidly developing technology, the latter can
actually outweigh the former.

(Besides, git in the stdlib doesn't make much sense :).

>Barry made bzr mirrors of the python svn tree. Not a python with bzr included.

Bingo.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/9ef0ed2b/attachment-0005.pgp>

From david.lyon at pythontest.org  Wed Jan 20 04:43:12 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 14:43:12 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
Message-ID: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>


> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

> Python 3 is still modularized, still has a standard library, etc. If
> you're really interested in helping with the standard library, get on
> stdlib-sig, and get ready to write code and PEPs.

Thank you for your direction to move these items forward to PEPs
and Code.

> i sincerely doubt that a source control system will be included in the
> standard library in the future. Especially 3.

Yeah and who twenty years ago thought you would get a 1GB memory
card for $3 when all we had was 10Meg hard disks and they were
the full 8" platter.

> A SCM is not a "package management system".

Exactly. It almost makes the need for a "package management system"
pretty much obsolete if you can update your code directly from
the developers sources.

That's what all these SCMs provide. Plus it's addictive. It's
hard to go back to 'package' style technology once you have
all your code on an SCM based feed.

> Barry was talking about mirrors of the python code. It is true a
> "package manager" could be developed based on a SCM, however you need
> to implement this far away from the stdlib and get traction with it
> within the community long before inclusion would be considered.

I think I'll have better chances with PEPs.

Being honest, if wonderful libraries like Sphinx and Mercurial
and Git and BZR can't make it into the stdlib, then there is
no hope for even newer code to get in there.

Plus, promoting all sorts of new and fangled tools however
good they may or may not be just confuses users and ends
up being a waste of time imho. It isn't good management
of volounteers time and effort.

If you could imagine disaster relief coordinated this
way, it would just be a disaster in itself.

That's why it has taken some 5 years to get PEP-345 done.

> The decision to move python's source control from SVN to mercurial was
> controversial enough; including 3 or more scm libraries into core
> would be an intractable uphill mountain of bike sheds.

Not at all.

It would be a very fair thing to do. Not to mention being
great for users.

> Barry made bzr mirrors of the python svn tree. Not a python with bzr
> included.

I can't resist asking for that again.. I heard it only in Monty
speek. Did you just say ?:

 "Barry made a bizarre mirror of the python suvern tree. Not a
  python with a buzzer included."

Anyway.. Maybe I do get what your talking about. Even if you do
talk with a strange accent. :-)

David






From jnoller at gmail.com  Wed Jan 20 05:10:07 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 19 Jan 2010 23:10:07 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4222a8491001192010v66b728dbxa7ad1ed1fc1df15f@mail.gmail.com>

On Tue, Jan 19, 2010 at 10:43 PM, David Lyon <david.lyon at pythontest.org> wrote:
[snip]
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.

Did you ever stop to think that some package authors do not want their
code in the standard library? That throwing random shiny things in
there just makes it a junk drawer?

Besides: Show us the PEP to include Sphinx, and it's dependencies in
the standard lib, with Georg's signature at the bottom.

The authors of modules have to want things to be in there, they have
to be best of breed, tested or common-enough patterns to warrant a
slot. We have things in the standard lib which still need more TLC and
maintenance, or they need to be shunted into space.

Adding random things, which may or may not help packaging, and
installing things from random places in someone source repository
won't make things better.

> Plus, promoting all sorts of new and fangled tools however
> good they may or may not be just confuses users and ends
> up being a waste of time imho. It isn't good management
> of volounteers time and effort.
>
> If you could imagine disaster relief coordinated this
> way, it would just be a disaster in itself.
>
> That's why it has taken some 5 years to get PEP-345 done.

I'm going to assume that you're trolling now, or intentionally
misrepresenting facts. Maybe a little of both. A PEP, and an
implementation and the ability to rationally debate, discuss and
defend your proposal is what is needed to enact changes on policies,
python-core or the standard library.

>> The decision to move python's source control from SVN to mercurial was
>> controversial enough; including 3 or more scm libraries into core
>> would be an intractable uphill mountain of bike sheds.
>
> Not at all.
>
> It would be a very fair thing to do. Not to mention being
> great for users.

There should be one-- and preferably only one --obvious way to do it.

> Anyway.. Maybe I do get what your talking about. Even if you do
> talk with a strange accent. :-)

My sense of humor has been disabled by repeated stunning at your
hands. I admire your enthusiasm, even if I do think some of it is
misplaced, or at guided into the proper channels at very least.

Please, you seem to have the time and willingness to help, please go
about this the right way. Discuss things on the proper lists, make
concrete proposals. If you have have standard lib changes, discuss
them on stdlib-sig, if you have ideas about python-the-language, or
the interpreter, etc - please discuss it on python-ideas.

jesse


From ben+python at benfinney.id.au  Wed Jan 20 05:16:25 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 15:16:25 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87wrzdfm1y.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
> and BZR can't make it into the stdlib, then there is no hope for even
> newer code to get in there.

Those are applications, not libraries. Applications don't belong in the
standard library.

-- 
 \     ?If you pick up a starving dog and make him prosperous, he will |
  `\      not bite you. This is the principal difference between a dog |
_o__)                    and a man.? ?Mark Twain, _Pudd'n'head Wilson_ |
Ben Finney



From brian.curtin at gmail.com  Wed Jan 20 05:19:47 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Tue, 19 Jan 2010 22:19:47 -0600
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <cf9f31f21001192019p25665318t3d99caf3db27198e@mail.gmail.com>

On Tue, Jan 19, 2010 at 21:43, David Lyon <david.lyon at pythontest.org> wrote:

>
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
>

I'm not entirely sure I see why the inclusion of a SCM into the stdlib is
necessary.

Just because pieces of software are mature and proven in their fields
doesn't mean we should add them, or that them *not* being in the stdlib
should be a basis for other projects making it in.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/3a2d80f8/attachment-0005.htm>

From barry at python.org  Wed Jan 20 05:26:44 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:26:44 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119232644.7fa25691@freewill>

On Jan 20, 2010, at 02:43 PM, David Lyon wrote:

>> On Tue, Jan 19, 2010 at 7:51 PM, Jesse Noller wrote:

>> A SCM is not a "package management system".
>
>Exactly. It almost makes the need for a "package management system"
>pretty much obsolete if you can update your code directly from
>the developers sources.
>
>That's what all these SCMs provide. Plus it's addictive. It's
>hard to go back to 'package' style technology once you have
>all your code on an SCM based feed.

Well...  I'm not so sure.  A package management system like apt does a /ton/
of additional bookkeeping and work to ensure a robust, highly consistent,
functioning system.  And while both Python and most Linux distributions have
their own notion of "package management", they don't always play nicely
together.  Tarek and the distutils-sig's work is trying to make the world a
better place by bridging this gap better, and there is code out there that
makes it easier to say import a Python package from the Cheeseshop and
.deb-ify it for use on Debian and Ubuntu.

There's also work being done in Launchpad that will allow you to
"build-from-branch" so that in a sense you could let a build farm take your
Bazaar branches and automatically build the packages from them.

I've strayed off-topic I suppose, but I see SCMs and package managers as
complementary technologies that help with important parts of the process of
delivering software to end-users, but I don't quite see how one can make the
other obsolete.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/4be8f756/attachment-0005.pgp>

From david.lyon at pythontest.org  Wed Jan 20 05:29:34 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 15:29:34 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>

> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>
> I'd be surprised if any of the big 3 DVCS developers would actually /want/
> their stuff in the stdlib.

If they ask, they'll get told they're motorbike-shedding. "It's better
if their users ask". So here I am as a user doing things the 'right'
way.

> Being in the stdlib has its advantages and
> disadvantages.  I think for rapidly developing technology, the
> latter can actually outweigh the former.

If it's about being able to do updates, then I think this
resolves an old and circular argument. As the SCM implementation
would, one would expect, to be able to update itself.

Side benefits are that it can update everything else along
with it at the same time. User Apps, Packages, whatever.

It's even better having SCM in an Industrial/Scientific
environment. Here's an example:

 - a machine breaks..  (I mean the software for/in it)

 - you fix the code, maybe on the spot

 - you commit and push back to the repository

 - your code gets checked in and run through the testbot
   and then you get blamed and have to do the whole thing
   again properly with a test case. Oh well..

Well anyway, whatever you guys might say, that's a whole
lot more efficient than running back to the development
machine and going through some obscure build and test
and publish process to do a fix on a production machine.

Point : The fact that SCMs are two way is great in
        a production environment. No packaging solution
        can come close.

So why not have python SCMs included as batteries in python..

All these arguments I can take off to the stdlib list when I get
the chance..

David




From barry at python.org  Wed Jan 20 05:50:36 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Jan 2010 23:50:36 -0500
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
Message-ID: <20100119235036.57f6e867@freewill>

Okay, last follow up on this and then I'm going to bed. :)

On Jan 20, 2010, at 03:29 PM, David Lyon wrote:

>> On Jan 19, 2010, at 08:09 PM, Barry Warsaw wrote:
>>
>> I'd be surprised if any of the big 3 DVCS developers would actually /want/
>> their stuff in the stdlib.
>
>If they ask, they'll get told they're motorbike-shedding. "It's better
>if their users ask". So here I am as a user doing things the 'right'
>way.

Actually, you're not.  It's not up to the Python community to initiate this.
If you really want this, you should engage with the relevant DVCS communities
and push them to request it.

>Side benefits are that it can update everything else along
>with it at the same time. User Apps, Packages, whatever.

I get that.  Heck, I still run one Gentoo server which I think is as close to
the edge you're describing as I'm comfortable with.  It's all great until the
wheels come off and then it can take *days* to get a functioning system
again.

The big difference is that I rely on my DVCS to keep one small thing, or a few
variants of the same thing, all sane.  But I rely on my distribution vendor to
keep a thousand complex, interdependent, interacting, sometimes conflicting
things sane and working.

>Point : The fact that SCMs are two way is great in
>        a production environment. No packaging solution
>        can come close.

Try talking with some hard-core operations guys, the folks with the keys to
the data centers, who work tireless, insanely hours keeping incredibly complex
systems running with very little downtime.  I think you'd get a different
perspective to put it mildly. :)

to-sleep-perchance-to-dream-ly y'rs,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100119/0def1cc9/attachment-0005.pgp>

From david.lyon at pythontest.org  Wed Jan 20 06:10:15 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 16:10:15 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <87wrzdfm1y.fsf@benfinney.id.au>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
Message-ID: <1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>

> "David Lyon" <david.lyon at pythontest.org> writes:
>
>> Being honest, if wonderful libraries like Sphinx and Mercurial and Git
>> and BZR can't make it into the stdlib, then there is no hope for even
>> newer code to get in there.
>
> Those are applications, not libraries. Applications don't belong in the
> standard library.

Haha funny..

Well using that logic, distutils is an application..

Are you saying that distutils should be removed? That
is most certainly an application.

Lets not get too pedantic here. Mercurial and bzr have a built
in API that can be called in a library like way. It's true they
also have a command line interface in the same way that distutils
does.

I'm not saying anything negative about distutils. Given that
Tarek has an upcoming Pycon presentation where the program
talks about a distutils revamp.

I'm hoping that he can find some young 20 yr olds and
put a cool web interface on that thing. Given that there
are empty sprints at pycon. It couldn't hurt to throw
that challenge out. Anyway, we'll see..


David




From ben+python at benfinney.id.au  Wed Jan 20 06:32:10 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:32:10 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
	<1669.218.214.45.58.1263961774.squirrel@syd-srv02.ezyreg.com>
	<20100119235036.57f6e867@freewill>
Message-ID: <87iqaxfijp.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> On Jan 20, 2010, at 03:29 PM, David Lyon wrote:
> >So here I am as a user doing things the 'right' way.
>
> Actually, you're not. It's not up to the Python community to initiate
> this. If you really want this, you should engage with the relevant
> DVCS communities and push them to request it.

Where ?push? must be strictly limited by a continual awareness that the
whole idea could just be bad.

If you find yourself in a tiny minority pushing for a change, it *could*
be that you have a great idea and the vast majority don't realise it
yet. But you must be realistic about the likelihood that the change is a
very *bad* idea, and frequently evaluate it for signs of that.

-- 
 \     ?I used to think that the brain was the most wonderful organ in |
  `\   my body. Then I realized who was telling me this.? ?Emo Philips |
_o__)                                                                  |
Ben Finney



From ben+python at benfinney.id.au  Wed Jan 20 06:34:07 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 Jan 2010 16:34:07 +1100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
	<87wrzdfm1y.fsf@benfinney.id.au>
	<1695.218.214.45.58.1263964215.squirrel@syd-srv02.ezyreg.com>
Message-ID: <87eillfigg.fsf@benfinney.id.au>

"David Lyon" <david.lyon at pythontest.org> writes:

> Well using that logic, distutils is an application..

Distutils is an application, the function of which is essential to
allowing sane development of Python packages. It's a special case. We
need to strictly limit the number of special cases, not gleefully add to
them.

-- 
 \        ?I'd take the awe of understanding over the awe of ignorance |
  `\                                          any day.? ?Douglas Adams |
_o__)                                                                  |
Ben Finney



From matthieu.brucher at gmail.com  Wed Jan 20 07:49:36 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 07:49:36 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>

> I'd be happy to see bzr and mercurial and git all made it together
> into the stdlib for python 3. That would give a superb updating
> mechanism for python that would propel python well beyond
> the dinosaur badlands of CPAN and other languages.

I think there are several points that make them not includable in Python:
- git is not written in Python
- bzr and mercurial have a life cycle much shorter than Python's, it's
the same issue than with other libraries where another community
develops them.

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From david.lyon at pythontest.org  Wed Jan 20 08:20:02 2010
From: david.lyon at pythontest.org (David Lyon)
Date: Wed, 20 Jan 2010 18:20:02 +1100 (EST)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
Message-ID: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>


Matthieu,

>> I'd be happy to see bzr and mercurial and git all made it together
>> into the stdlib for python 3. That would give a superb updating
>> mechanism for python that would propel python well beyond
>> the dinosaur badlands of CPAN and other languages.
>
> I think there are several points that make them not includable in Python:
> - git is not written in Python
> - bzr and mercurial have a life cycle much shorter than Python's, it's
> the same issue than with other libraries where another community
> develops them.

That's only two points. :-)

On 1; If that's true, I won't mention git again.

On 2; Who knows what their life cycle is. CVS is pretty much
      dead, and svn looks like it is on the way out.
      I can't think of how anything could be better than
      mercurial or bzr but I know I will be proved wrong.

At the end of the day, we are making a decision about whether
the language is 'set-in-stone' or whether it is still
evolving.

To me, Python 1.x had it's own distinct "era", as has
Python 2.x

Hoping that the Python 3 era can be a little more flexible
and perphaps "cleaner" than the 2.x era is all that I am
thinking here.

Have a nice day

David





From matthieu.brucher at gmail.com  Wed Jan 20 09:05:19 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Wed, 20 Jan 2010 09:05:19 +0100
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <e76aa17f1001200005j6344672fo99826cc2e8648141@mail.gmail.com>

> That's only two points. :-)

In French, we say that several starts with 2 ;)

> On 1; If that's true, I won't mention git again.

I tis, you can check on the git repository (it's a mix of C, perl,
shell scripts, Python, ...)

> On 2; Who knows what their life cycle is.

You can check on their websites, their cycles are far shorter than
Python minor releases (several months vs several years).

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From stephen at xemacs.org  Wed Jan 20 11:22:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 20 Jan 2010 19:22:55 +0900
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <20100119223230.4c4a7ed5@freewill>
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<20100119223230.4c4a7ed5@freewill>
Message-ID: <87aaw9gjnk.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > (Besides, git in the stdlib doesn't make much sense :).

"Dulwich."


From solipsis at pitrou.net  Wed Jan 20 12:28:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Jan 2010 11:28:16 +0000 (UTC)
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
References: <20100119081639.5d431ed9@freewill>
	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>
	<20100119185439.299660c7@freewill>
	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>
	<4222a8491001191709m38f1cb96if5e546ed7dc008ab@mail.gmail.com>
	<1622.218.214.45.58.1263958992.squirrel@syd-srv02.ezyreg.com>
Message-ID: <loom.20100120T122742-162@post.gmane.org>

David Lyon <david.lyon <at> pythontest.org> writes:
> 
> I think I'll have better chances with PEPs.
> 
> Being honest, if wonderful libraries like Sphinx and Mercurial
> and Git and BZR can't make it into the stdlib, then there is
> no hope for even newer code to get in there.
> [snip]

This is python-ideas material, can you take it there? Thank you.

Antoine.




From ncoghlan at gmail.com  Wed Jan 20 13:16:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Jan 2010 22:16:34 +1000
Subject: [Python-Dev] Bazaar branches available (again) on Launchpad
In-Reply-To: <47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
References: <20100119081639.5d431ed9@freewill>	<1377.218.214.45.58.1263943004.squirrel@syd-srv02.ezyreg.com>	<20100119185439.299660c7@freewill>	<1488.218.214.45.58.1263948683.squirrel@syd-srv02.ezyreg.com>	<e76aa17f1001192249x295d6422u61aec8d3e0576da4@mail.gmail.com>
	<47475.115.128.47.203.1263972002.squirrel@syd-srv02.ezyreg.com>
Message-ID: <4B56F422.5010607@gmail.com>

David Lyon wrote:
> On 2; Who knows what their life cycle is. CVS is pretty much
>       dead, and svn looks like it is on the way out.
>       I can't think of how anything could be better than
>       mercurial or bzr but I know I will be proved wrong.

I believe you misunderstood what Matthieu meant by life cycle there:
think "release cycle". If a project pushes out new releases
significantly more often than every 18-24 months (as is currently true
for all of the major SCM tools), then that fact alone makes it a very
bad fit for the Python standard library.

And centralised source control will be going strong for years. The DVCS
approach may be great for the open source world, but the gains are far
more limited in a closed source shop (especially a group writing
internal corporate applications which doesn't need to keep many, if any,
maintenance branches going).

If we weren't dealing with 4 active branches, the DVCS discussion would
have got a lot less traction with the core developers - aside from
better handling of multiple lines of development, most of the benefits
of the switch to a DVCS accrue to people without commit access to the
SVN repository.

Anyway, we've wandered far afield from legit python-dev topics now. Any
further ideas about super_mega_easy_install functionality that can pull
code from source control systems and build it rather than requiring
prebuild source tarballs should be directed to python-ideas (they
probably need to bake more even before they make an appearance on
distutils-sig).

Cheers,
Nick.

P.S. As Jesse said... your enthusiasm is great, but please don't assume
that some inherent conservatism on the part of other developers is
automatically evil or the result of a failure to see your point. A lot
of people around the world rely on our stuff every day. We owe it to
them to be measured in our actions and to put serious thought into any
major changes or additions we make to the language and the standard
library. For the current stage of its development, Python 3 is in a good
place from our point of view - its major carrot has really always been
the better Unicode support it offers, and the ever-increasing
globalisation of the web will create more and more pressure pushing
developers in that direction as the years go by. Sure, Python 3 cleans
up assorted other things as well, but the change to the text processing
model is the big one that is fundamentally incompatible with the
architecture of the 2.x series. Compared to that change, everything else
is just tinkering.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------