Mailman 3 January 2006 - Python-Dev

str with base
by Alex Martelli 15 Mar '06

15 Mar '06

Is it finally time in Python 2.5 to allow the "obvious" use of, say, str(5,2) to give '101', just the converse of the way int('101',1) gives 5? I'm not sure why str has never allowed this obvious use -- any bright beginner assumes it's there and it's awkward to explain why it's not!-). I'll be happy to propose a patch if the BDFL blesses this, but I don't even think it's worth a PEP... it's an inexplicable though long-standing omission (given the argumentative nature of this crowd I know I'll get pushback, but I still hope the BDFL can Pronounce about it anyway;-). Alex

42 134

PythonCore\CurrentVersion
by "Martin v. Löwis" 13 Mar '06

13 Mar '06

What happened to the CurrentVersion registry entry documented at http://www.python.org/windows/python/registry.html AFAICT, even the python15.wse file did not fill a value in this entry (perhaps I'm misinterpreting the wse file, though). So was this ever used? Why is it documented, and who documented it (unfortunately, registry.html is not in cvs/subversion, either)? Regards, Martin

5 11

fixing log messages
by Fredrik Lundh 09 Mar '06

09 Mar '06

just noticed an embarrasing misspelling in one of my recent checkins, only to find that I cannot fix it: $ svn propedit --revprop -r 41759 svn:log svn: Repository has not been enabled to accept revision propchanges; ask the administrator to create a pre-revprop-change hook $ would it be a good idea to ask the administrator to do this ? </F>

4 8

When will regex really go away?
by skip＠pobox.com 03 Mar '06

03 Mar '06

I ran Fredrik's listmodules script in my current sandbox and got a deprecation warning for the regex module. According to PEP 4 it is already obsolete. I saw nothing there about the timeframe for actual removal. Will it ever go away? Skip

5 6

problem with genexp
by Neal Norwitz 25 Feb '06

25 Feb '06

There's a problem with genexp's that I think really needs to get fixed. See http://python.org/sf/1167751 the details are below. This code: >>> foo(a = i for i in range(10)) generates "NameError: name 'i' is not defined" when run because: 2 0 LOAD_GLOBAL 0 (foo) 3 LOAD_CONST 1 ('a') 6 LOAD_GLOBAL 1 (i) 9 CALL_FUNCTION 256 12 POP_TOP 13 LOAD_CONST 0 (None) 16 RETURN_VALUE If you add parens around the code: foo(a = i for i in range(10)) You get something quite different: 2 0 LOAD_GLOBAL 0 (foo) 3 LOAD_CONST 1 ('a') 6 LOAD_CONST 2 (<code object <generator expression> at 0x2a960baae8, file "<stdin>", line 2>) 9 MAKE_FUNCTION 0 12 LOAD_GLOBAL 1 (range) 15 LOAD_CONST 3 (10) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 CALL_FUNCTION 256 28 POP_TOP 29 LOAD_CONST 0 (None) 32 RETURN_VALUE I agree with the bug report that the code should either raise a SyntaxError or do the right thing. n

5 10

file.next() vs. file.readline()
by Thomas Wouters 11 Feb '06

11 Feb '06

Twice now, I've been bitten by the 'mixing file-iteration with readline' issue. (Yeah, twice.. Good thing I don't write anything important in Python ;) I've also seen a number of newbies bitten by the same thing. The issue, for those that aren't aware, is that when mixing file-iteration and readline (or readlines or such), you run the risk of getting data in the wrong order. This is because file-iteration uses an 8k buffer, while readline doesn't. It's not really a big deal, once you understand it's not wise to mix iteration and the readline(s) methods. I do wonder, though, why Python doesn't take care to raise an exception when readline is called with 'stuff' in the iteration-buffer. A cursory look at the source doesn't reveal any glaring problems. It's a single check, possibly two, with good locality of reference. Also, raising an exception when there is stuff in the buffer would only make mixing iteration/readline an error when you would actually, in fact, lose or mess up data. In other words, it would only raise exceptions for existing cases that are already broken. Is there something I've missed that makes the check undesireable or unfeasible? Or should I just assume no on has gotten around to it (or could be bothered), and just submit a patch? :) (Making readline and friends use the same buffer may seem the better solution to some, but I'm sure there's a whole discussion behind that, about whether to buffer or not. All non-iteration routines in fileobject.c take pretty good care not to read too much, and I choose to see that as explicitly designed that way.) Absent-ly y'rs, -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

2 3

[PATCH] Fix dictionary subclass semantics when used as global dictionaries
by Crutcher Dunnavant 06 Feb '06

06 Feb '06

There is an inconsistancy in the way that dictionary subclasses behave when they are used as as namespaces in execs. Basically, while python 2.4 permits the usage of dictionary subclasses for local environments, it still bypasses the subclass functions and uses the C API for global environments. The attached patch (and unittest!) addresses this issue. I'm pretty sure we keep the fast path in this. -- Crutcher Dunnavant <crutcher(a)gmail.com> monket.samedi-studios.com

5 11

The path module PEP
by BJörn Lindqvist 02 Feb '06

02 Feb '06

The last time this was discussed six months ago it seemed like most of python-dev fancied Jason Orendorff's path module. But Guido wanted a PEP and noone created one. So I decided to claim the fame and write one since I also love the path module. :) Much of it is copy-pasted from Peter Astrand's PEP 324, many thanks to him. ################################################# PEP: XXX Title: path - Object oriented handling of filesystem paths Version: $Revision: XXXX $ Last-Modified: $Date: 2006-01-24 19:24:59 +0100 (Sat, 29 Jan 2005) $ Author: Björn Lindqvist <bjourne(a)gmail.com> Status: Dummo Type: Standards Track (library) Created: 24-Jan-2006 Content-Type: text/plain Python-Version: 2.4 Abstract This PEP describes a new class for handling paths in an object oriented fashion. Motivation Dealing with filesystem paths is a common task in any programming language, and very common in a high-level language like Python. Good support for this task is needed, because: - Almost every program uses paths to access files. It makes sense that a task, that is so often performed, should be as intuitive and as easy to perform as possible. - It makes Python an even better replacement language for over-complicated shell scripts. Currently, Python has a large number of different functions scattered over half a dozen modules for handling paths. This makes it hard for newbies and experienced developers to to choose the right method. The Path class provides the following enhancements over the current common practice: - One "unified" object provides all functionality from previous functions. - Subclassability - the Path object can be extended to support paths other than filesystem paths. The programmer does not need to learn a new API, but can reuse his or her knowledge of Path to deal with the extended class. - With all related functionality in one place, the right approach is easier to learn as one does not have to hunt through many different modules for the right functions. - Python is an object oriented language. Just like files, datetimes and sockets are objects so are paths, they are not merely strings to be passed to functions. Path objects are inherently a pythonic idea. - Path takes advantage of properties. Properties make for more readable code. if imgpath.ext == 'jpg': jpegdecode(imgpath) Is better than: if os.path.splitexit(imgpath)[1] == 'jpg': jpegdecode(imgpath) Rationale The following points summarizes the design: - Path extends from string, therefore all code which expects string pathnames need not be modified and no existing code will break. - A Path object can be created either by using the classmethod Path.cwd, by instantiating the class with a string representing a path or by using the default constructor which is equivalent with Path("."). - The factory functions in popen2 have been removed, because I consider the class constructor equally easy to work with. - Path provides for common pathname manipulation, pattern expansion, pattern matching and other high-level file operations including copying. Basically everything path related except for manipulation of file contents which file objects are better suited for. - Platform incompatibilites are dealt with by not instantiating system specific methods. Specification This class defines the following public methods: # Special Python methods. def __new__(cls, init = os.curdir): ... def __repr__(self): ... def __add__(self, more): ... def __radd__(self, other): ... def __div__(self, rel): ... def __truediv__(self, rel): ... # Alternative constructor. def cwd(cls): ... # Operations on path strings. def abspath(sef): ... def normcase(self): ... def normpath(self): ... def realpath(self): ... def expanduser(self): ... def expandvars(self): ... def dirname(self): ... def basename(self): ... def expand(self): ... def splitpath(self): ... def splitdrive(self): ... def splitext(self): ... def stripext(self): ... def splitunc(self): ... [1] def joinpath(self, *args): ... def splitall(self): ... def relpath(self): ... def relpathto(self, dest): ... # Properties about the path. parent, name, namebase, ext, drive, uncshare[1] # Operations that return lists of paths. def listdir(self, pattern = None): ... def dirs(self, pattern = None): ... def files(self, pattern = None): ... def walk(self, pattern = None): ... def walkdirs(self, pattern = None): ... def walkfiles(self, pattern = None): ... def match(self, pattern): def matchcase(self, pattern): def glob(self, pattern): # Methods for retrieving information about the filesystem # path. def exists(self): ... def isabs(self): ... def isdir(self): ... def isfile(self): ... def islink(self): ... def ismount(self): ... def samefile(self, other): ... [1] def getatime(self): ... def getmtime(self): ... def getctime(self): ... def getsize(self): ... def access(self, mode): ... [1] def stat(self): ... def lstat(self): ... def statvfs(self): ... [1] def pathconf(self, name): ... [1] def utime(self, times): ... def chmod(self, mode): ... def chown(self, uid, gid): ... [1] def rename(self, new): ... def renames(self, new): # Filesystem properties for path. atime, getmtime, getctime, size # Methods for manipulating information about the filesystem # path. def utime(self, times): ... def chmod(self, mode): ... def chown(self, uid, gid): ... [1] def rename(self, new): ... def renames(self, new): ... # Create/delete operations on directories def mkdir(self, mode = 0777): ... def makedirs(self, mode = 0777): ... def rmdir(self): ... def removedirs(self): ... # Modifying operations on files def touch(self): ... def remove(self): ... def unlink(self): ... # Modifying operations on links def link(self, newpath): ... def symlink(self, newlink): ... def readlink(self): ... def readlinkabs(self): ... # High-level functions from shutil def copyfile(self, dst): ... def copymode(self, dst): ... def copystat(self, dst): ... def copy(self, dst): ... def copy2(self, dst): ... def copytree(self, dst, symlinks = True): ... def move(self, dst): ... def rmtree(self, ignore_errors = False, onerror = None): ... # Special stuff from os def chroot(self): ... [1] def startfile(self): ... [1] [1] - Method is not availible on all platforms. Replacing older functions with the Path class In this section, "a ==> b" means that b can be used as a replacement for a. In the following examples, we assume that the Path class is imported with "from path import Path". Replacing os.path.join ---------------------- os.path.join(os.getcwd(), "foobar") ==> Path.cwd() / "foobar" Replacing os.path.splitext -------------------------- os.path.splitext("Python2.4.tar.gz")[1] ==> Path("Python2.4.tar.gz").ext Replacing glob.glob ------------------- glob.glob("/lib/*.so") ==> Path("/lib").glob("*.so") Deprecations Introducing this module to the standard library introduces the need to deprecate a number of existing modules and functions. The table below explains which existing functionality that must be deprecated. PATH METHOD DEPRECATES FUNCTION normcase() os.path.normcase() normpath() os.path.normpath() realpath() os.path.realpath() expanduser() os.path.expanduser() expandvars() os.path.expandvars() dirname() os.path.dirname() basename() os.path.basename() splitpath() os.path.split() splitdrive() os.path.splitdrive() splitext() os.path.splitext() splitunc() os.path.splitunc() joinpath() os.path.join() listdir() os.listdir() [fnmatch.filter()] match() fnmatch.fnmatch() matchcase() fnmatch.fnmatchcase() glob() glob.glob() exists() os.path.exists() isabs() os.path.isabs() isdir() os.path.isdir() isfile() os.path.isfile() islink() os.path.islink() ismount() os.path.ismount() samefile() os.path.samefile() getatime() os.path.getatime() getmtime() os.path.getmtime() getsize() os.path.getsize() cwd() os.getcwd() access() os.access() stat() os.stat() lstat() os.lstat() statvfs() os.statvfs() pathconf() os.pathconf() utime() os.utime() chmod() os.chmod() chown() os.chown() rename() os.rename() renames() os.renames() mkdir() os.mkdir() makedirs() os.makedirs() rmdir() os.rmdir() removedirs() os.removedirs() remove() os.remove() unlink() os.unlink() link() os.link() symlink() os.symlink() readlink() os.readlink() chroot() os.chroot() startfile() os.startfile() copyfile() shutil.copyfile() copymode() shutil.copymode() copystat() shutil.copystat() copy() shutil.copy() copy2() shutil.copy2() copytree() shutil.copytree() move() shutil.move() rmtree() shutil.rmtree() The Path class deprecates the whole of os.path, shutil, fnmatch and glob. A big chunk of os is also deprecated. Open Issues Some functionality of Jason Orendorff's path module have been omitted: * Function for opening a path - better handled by the builtin open(). * Functions for reading and writing a whole file - better handled by file objects read() and write() methods. * A chdir() function may be a worthy inclusion. * A deprecation schedule needs to be setup. How much functionality should Path implement? How much of existing functionality should it deprecate and when? * Where should the class be placed and what should it be called? The functions and modules that this new module is trying to replace (os.path, shutil, fnmatch, glob and parts of os are expected to be available in future Python versions for a long time, to preserve backwards compatibility. Reference Implementation Currently, the Path class is implemented as a thin wrapper around the standard library modules sys, os, fnmatch, glob and shutil. The intention of this PEP is to move functionality from the aforementioned modules to Path while they are being deprecated. For more detail, and diverging implementations see: * http://www.jorendorff.com/articles/python/path/path.py * http://svn.python.org/projects/sandbox/trunk/path/path.py * http://cafepy.com/quixote_extras/rex/path/enhpath.py Examples In this section, "a ==> b" means that b can be used as a replacement for a. 1. Make all python files in the a directory executable: DIR = '/usr/home/guido/bin' for f in os.listdir(DIR): if f.endswith('.py'): path = os.path.join(DIR, f) os.chmod(path, 0755) ==> for f in Path('/usr/home/guido/bin'): f.chmod(0755) 2. Delete emacs backup files: def delete_backups(arg, dirname, names): for name in names: if name.endswith('~'): os.remove(os.path.join(dirname, name)) ==> d = Path(os.environ['HOME']) for f in d.walkfiles('*~'): f.remove() 3. Finding the relative path to a file: b = Path('/users/peter/') a = Path('/users/peter/synergy/tiki.txt') a.relpathto(b) 4. Splitting a path into directory and filename: os.path.split("/path/to/foo/bar.txt") ==> Path("/path/to/foo/bar.txt").splitpath() 5. List all Python scripts in the current directory tree: list(Path().walkfiles("*.py")) 6. Create directory paths: os.path.join("foo", "bar", "baz") ==> Path("foo") / "bar" / "baz" Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -- mvh Björn

27 84

Compiler warnings
by Thomas Wouters 02 Feb '06

02 Feb '06

I noticed a few compiler warnings, when I compile Python on my amd64 with gcc 4.0.3: Objects/longobject.c: In function ‘PyLong_AsDouble’: Objects/longobject.c:655: warning: ‘e’ may be used uninitialized in this function Objects/longobject.c: In function ‘long_true_divide’: Objects/longobject.c:2263: warning: ‘aexp’ may be used uninitialized in this function Objects/longobject.c:2263: warning: ‘bexp’ may be used uninitialized in this function Modules/linuxaudiodev.c: In function ‘lad_obuffree’: Modules/linuxaudiodev.c:392: warning: ‘ssize’ may be used uninitialized in this function Modules/linuxaudiodev.c: In function ‘lad_bufsize’: Modules/linuxaudiodev.c:348: warning: ‘ssize’ may be used uninitialized in this function Modules/linuxaudiodev.c: In function ‘lad_obufcount’: Modules/linuxaudiodev.c:369: warning: ‘ssize’ may be used uninitialized in this function Those are all fairly harmless, just the compiler can't figure out that they are never actually used when they aren't explicitly initialized, because the initialization happens a few functioncalls deeper into the callstack. This one: Python/marshal.c: In function ‘PyMarshal_ReadShortFromFile’: Python/marshal.c:410: warning: ‘rf.end’ is used uninitialized in this function Python/marshal.c:410: warning: ‘rf.ptr’ is used uninitialized in this function (The linenumber is off, it should be 884) is more of the same, except you can't tell from the function that it is a "can never happen" situation: if PyMarshal_ReadShortFromFile() was called with NULL as fp, it would actually use the uninitialized 'end' and 'ptr' struct members. An assert() is probably in place here. Should these warnings be fixed? I know Tim has always argued to fix them, in the past (and I agree,) and it doesn't look like doing so, by initializing the variables, wouldn't be too big a performance hit. I also noticed test_logging is spuriously failing, and not just on my machine (according to buildbot logs.) Is anyone (Vinay?) looking at that yet? -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

12 29

Extension to ConfigParser
by Fuzzyman 01 Feb '06

01 Feb '06

Hello all, In the past there has been some discussion about a new module to replace ConfigParser. Most notably at http://wiki.python.org/moin/ConfigParserShootout Specific issues that could be addressed include : * Simpler API * Nested subsections * List values * Storing/converting datatypes * Config file schema * Keeps track of order of values Plus other issues. I'm the (co-)author of ConfigObj - http://www.voidspace.org.uk/python/configobj.html This is a reasonably mature project (now in it's fourth incarnation), and is being used in projects like `Bazaar <http://www.bazaar-ng.org/>`_ and `PlanetPlus <http://planetplus.python-hosting.com/>`_. It occurs to me that the ConfigObj API and syntax is *almost* fully compatible with ConfigParser. It would be possible to extend to the ConfigObj API to be backwards compatible with ConfigParser. This would bring the added benefits of ConfigObj, without needing to add an extra module to the standard library. Well nearly. ConfigObj supports config file schema with (optional) type conversion, through a companion module called validate. This could be included or left as an added option. Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and if accepted, do the work - which is not inconsiderable). Summary of ConfigObj ================ ConfigObj is a Python 2.2 compatible config file parser. It's major feature is simplicity of use. It reads (and writes) INI file like config files and presents the members using a dictionary interface. The order of keys/sections is preserved, and it allows nested subsections to any level : e.g. :: key = value [section] key = value [[sub-section]] key = value It is fully documented with a barrage of doctests. All comments in the config file are also preserved as attributes of the object, and will be written back out. This can be useful for including comments in programatically generated config files. It is integrated with a powerful validation system. Difficulties & Differences ================= A ConfigObj instance is a sub-class of the dictionary datatpe. This means that the ``get`` method of ConfigParser clashes. ConfigObj allows values in the root section (why not ?). ConfigObj doesn't support line continuations (it does all multi-line values through the use of triple quoted strings). ConfigObj currently only allows '=' as a valid divider. Creating ConfigParser (and related classes) compatibility is a big job. Solution ===== All of these problems (if deemed necessary) can be resolved. Either through options, or just by extending the ConfigObj API. I'm happy to put the work in. Comments ? All the best, Fuzzyman http://www.voidspace.org.uk/python/index.shtml

15 32