Mailman 3 March 2000 - Python-Dev

Should None be a keyword?
by Ka-Ping Yee March 17, 2000

March 17, 2000

Related to my last message: should None become a keyword in Py3K? -- ?!ng

5 4

Unicode Update 2000-03-17
by M.-A. Lemburg March 17, 2000

March 17, 2000

Attached you find an update of the Unicode implementation. The patch is against the current CVS version. I would appreciate if someone with CVS checkin permissions could check the changes in. The patch contains all bugs and patches sent this week and also fixes a leak in the codecs code and a bug in the free list code for Unicode objects (which only shows up when compiling Python with Py_DEBUG; thanks to MarkH for spotting this one). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ Only in CVS-Python/Doc/tools: anno-api.py diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Include/unicodeobject.h Python+Unicode/Include/unicodeobject.h --- CVS-Python/Include/unicodeobject.h Fri Mar 17 15:24:30 2000 +++ Python+Unicode/Include/unicodeobject.h Tue Mar 14 10:38:08 2000 @@ -1,8 +1,5 @@ #ifndef Py_UNICODEOBJECT_H #define Py_UNICODEOBJECT_H -#ifdef __cplusplus -extern "C" { -#endif /* @@ -109,8 +106,9 @@ /* --- Internal Unicode Operations ---------------------------------------- */ /* If you want Python to use the compiler's wctype.h functions instead - of the ones supplied with Python, define WANT_WCTYPE_FUNCTIONS. - This reduces the interpreter's code size. */ + of the ones supplied with Python, define WANT_WCTYPE_FUNCTIONS or + configure Python using --with-ctype-functions. This reduces the + interpreter's code size. */ #if defined(HAVE_USABLE_WCHAR_T) && defined(WANT_WCTYPE_FUNCTIONS) @@ -169,6 +167,10 @@ (!memcmp((string)->str + (offset), (substring)->str,\ (substring)->length*sizeof(Py_UNICODE))) +#ifdef __cplusplus +extern "C" { +#endif + /* --- Unicode Type ------------------------------------------------------- */ typedef struct { @@ -647,7 +649,7 @@ int direction /* Find direction: +1 forward, -1 backward */ ); -/* Count the number of occurances of substr in str[start:end]. */ +/* Count the number of occurrences of substr in str[start:end]. */ extern DL_IMPORT(int) PyUnicode_Count( PyObject *str, /* String */ @@ -656,7 +658,7 @@ int end /* Stop index */ ); -/* Replace at most maxcount occurances of substr in str with replstr +/* Replace at most maxcount occurrences of substr in str with replstr and return the resulting Unicode object. */ extern DL_IMPORT(PyObject *) PyUnicode_Replace( diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/codecs.py Python+Unicode/Lib/codecs.py --- CVS-Python/Lib/codecs.py Sat Mar 11 00:20:43 2000 +++ Python+Unicode/Lib/codecs.py Mon Mar 13 14:33:54 2000 @@ -55,7 +55,7 @@ """ def encode(self,input,errors='strict'): - """ Encodes the object intput and returns a tuple (output + """ Encodes the object input and returns a tuple (output object, length consumed). errors defines the error handling to apply. It defaults to diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/encodings/__init__.py Python+Unicode/Lib/encodings/__init__.py --- CVS-Python/Lib/encodings/__init__.py Sat Mar 11 00:17:18 2000 +++ Python+Unicode/Lib/encodings/__init__.py Mon Mar 13 14:30:33 2000 @@ -30,13 +30,13 @@ import string,codecs,aliases _cache = {} -_unkown = '--unkown--' +_unknown = '--unknown--' def search_function(encoding): # Cache lookup - entry = _cache.get(encoding,_unkown) - if entry is not _unkown: + entry = _cache.get(encoding,_unknown) + if entry is not _unknown: return entry # Import the module diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/test_string.py Python+Unicode/Lib/test/test_string.py --- CVS-Python/Lib/test/test_string.py Sat Mar 11 10:52:43 2000 +++ Python+Unicode/Lib/test/test_string.py Mon Mar 13 10:12:46 2000 @@ -143,6 +143,7 @@ test('translate', 'xyz', 'xyz', table) test('replace', 'one!two!three!', 'one@two!three!', '!', '@', 1) +test('replace', 'one!two!three!', 'onetwothree', '!', '') test('replace', 'one!two!three!', 'one@two@three!', '!', '@', 2) test('replace', 'one!two!three!', 'one@two@three@', '!', '@', 3) test('replace', 'one!two!three!', 'one@two@three@', '!', '@', 4) diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/test_unicode.py Python+Unicode/Lib/test/test_unicode.py --- CVS-Python/Lib/test/test_unicode.py Fri Mar 17 15:24:31 2000 +++ Python+Unicode/Lib/test/test_unicode.py Mon Mar 13 10:13:05 2000 @@ -108,6 +108,7 @@ test('translate', u'xyz', u'xyz', table) test('replace', u'one!two!three!', u'one@two!three!', u'!', u'@', 1) +test('replace', u'one!two!three!', u'onetwothree', '!', '') test('replace', u'one!two!three!', u'one@two@three!', u'!', u'@', 2) test('replace', u'one!two!three!', u'one@two@three@', u'!', u'@', 3) test('replace', u'one!two!three!', u'one@two@three@', u'!', u'@', 4) diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Misc/unicode.txt Python+Unicode/Misc/unicode.txt --- CVS-Python/Misc/unicode.txt Sat Mar 11 00:14:11 2000 +++ Python+Unicode/Misc/unicode.txt Fri Mar 17 16:55:11 2000 @@ -743,8 +743,9 @@ stream codecs as available through the codecs module should be used. -XXX There should be a short-cut open(filename,mode,encoding) available which - also assures that mode contains the 'b' character when needed. +The codecs module should provide a short-cut open(filename,mode,encoding) +available which also assures that mode contains the 'b' character when +needed. File/Stream Input: @@ -810,6 +811,10 @@ Introduction to Unicode (a little outdated by still nice to read): http://www.nada.kth.se/i18n/ucs/unicode-iso10646-oview.html +For comparison: + Introducing Unicode to ECMAScript -- + http://www-4.ibm.com/software/developer/library/internationalization-suppor… + Encodings: Overview: @@ -832,7 +837,7 @@ History of this Proposal: ------------------------- -1.2: +1.2: Removed POD about codecs.open() 1.1: Added note about comparisons and hash values. Added note about case mapping algorithms. Changed stream codecs .read() and .write() method to match the standard file-like object methods diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Modules/stropmodule.c Python+Unicode/Modules/stropmodule.c --- CVS-Python/Modules/stropmodule.c Wed Mar 1 10:22:53 2000 +++ Python+Unicode/Modules/stropmodule.c Mon Mar 13 14:33:23 2000 @@ -1054,7 +1054,7 @@ strstr replacement for arbitrary blocks of memory. - Locates the first occurance in the memory pointed to by MEM of the + Locates the first occurrence in the memory pointed to by MEM of the contents of memory pointed to by PAT. Returns the index into MEM if found, or -1 if not found. If len of PAT is greater than length of MEM, the function returns -1. diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Objects/stringobject.c Python+Unicode/Objects/stringobject.c --- CVS-Python/Objects/stringobject.c Tue Mar 14 00:14:17 2000 +++ Python+Unicode/Objects/stringobject.c Mon Mar 13 14:33:24 2000 @@ -1395,7 +1395,7 @@ strstr replacement for arbitrary blocks of memory. - Locates the first occurance in the memory pointed to by MEM of the + Locates the first occurrence in the memory pointed to by MEM of the contents of memory pointed to by PAT. Returns the index into MEM if found, or -1 if not found. If len of PAT is greater than length of MEM, the function returns -1. @@ -1578,7 +1578,7 @@ return NULL; if (sub_len <= 0) { - PyErr_SetString(PyExc_ValueError, "empty replacement string"); + PyErr_SetString(PyExc_ValueError, "empty pattern string"); return NULL; } new_s = mymemreplace(str,len,sub,sub_len,repl,repl_len,count,&out_len); Only in CVS-Python/Objects: stringobject.c.orig diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Objects/unicodeobject.c Python+Unicode/Objects/unicodeobject.c --- CVS-Python/Objects/unicodeobject.c Tue Mar 14 00:14:17 2000 +++ Python+Unicode/Objects/unicodeobject.c Wed Mar 15 10:49:19 2000 @@ -83,7 +83,7 @@ all objects on the free list having a size less than this limit. This reduces malloc() overhead for small Unicode objects. - At worse this will result in MAX_UNICODE_FREELIST_SIZE * + At worst this will result in MAX_UNICODE_FREELIST_SIZE * (sizeof(PyUnicodeObject) + STAYALIVE_SIZE_LIMIT + malloc()-overhead) bytes of unused garbage. @@ -180,7 +180,7 @@ unicode_freelist = *(PyUnicodeObject **)unicode_freelist; unicode_freelist_size--; unicode->ob_type = &PyUnicode_Type; - _Py_NewReference(unicode); + _Py_NewReference((PyObject *)unicode); if (unicode->str) { if (unicode->length < length && _PyUnicode_Resize(unicode, length)) { @@ -199,16 +199,19 @@ unicode->str = PyMem_NEW(Py_UNICODE, length + 1); } - if (!unicode->str) { - PyMem_DEL(unicode); - PyErr_NoMemory(); - return NULL; - } + if (!unicode->str) + goto onError; unicode->str[length] = 0; unicode->length = length; unicode->hash = -1; unicode->utf8str = NULL; return unicode; + + onError: + _Py_ForgetReference((PyObject *)unicode); + PyMem_DEL(unicode); + PyErr_NoMemory(); + return NULL; } static @@ -224,7 +227,6 @@ *(PyUnicodeObject **)unicode = unicode_freelist; unicode_freelist = unicode; unicode_freelist_size++; - _Py_ForgetReference(unicode); } else { free(unicode->str); @@ -489,7 +491,7 @@ } else { PyErr_Format(PyExc_ValueError, - "UTF-8 decoding error; unkown error handling code: %s", + "UTF-8 decoding error; unknown error handling code: %s", errors); return -1; } @@ -611,7 +613,7 @@ else { PyErr_Format(PyExc_ValueError, "UTF-8 encoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -733,7 +735,7 @@ } else { PyErr_Format(PyExc_ValueError, - "UTF-16 decoding error; unkown error handling code: %s", + "UTF-16 decoding error; unknown error handling code: %s", errors); return -1; } @@ -921,7 +923,7 @@ else { PyErr_Format(PyExc_ValueError, "Unicode-Escape decoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1051,6 +1053,10 @@ */ +static const Py_UNICODE *findchar(const Py_UNICODE *s, + int size, + Py_UNICODE ch); + static PyObject *unicodeescape_string(const Py_UNICODE *s, int size, @@ -1069,9 +1075,6 @@ p = q = PyString_AS_STRING(repr); if (quotes) { - static const Py_UNICODE *findchar(const Py_UNICODE *s, - int size, - Py_UNICODE ch); *p++ = 'u'; *p++ = (findchar(s, size, '\'') && !findchar(s, size, '"')) ? '"' : '\''; @@ -1298,7 +1301,7 @@ else { PyErr_Format(PyExc_ValueError, "Latin-1 encoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1369,7 +1372,7 @@ else { PyErr_Format(PyExc_ValueError, "ASCII decoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1431,7 +1434,7 @@ else { PyErr_Format(PyExc_ValueError, "ASCII encoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1502,7 +1505,7 @@ else { PyErr_Format(PyExc_ValueError, "charmap decoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1618,7 +1621,7 @@ else { PyErr_Format(PyExc_ValueError, "charmap encoding error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } @@ -1750,7 +1753,7 @@ else { PyErr_Format(PyExc_ValueError, "translate error; " - "unkown error handling code: %s", + "unknown error handling code: %s", errors); return -1; } diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Python/codecs.c Python+Unicode/Python/codecs.c --- CVS-Python/Python/codecs.c Fri Mar 10 23:57:27 2000 +++ Python+Unicode/Python/codecs.c Wed Mar 15 11:27:54 2000 @@ -93,9 +93,14 @@ PyObject *_PyCodec_Lookup(const char *encoding) { - PyObject *result, *args = NULL, *v; + PyObject *result, *args = NULL, *v = NULL; int i, len; + if (_PyCodec_SearchCache == NULL || _PyCodec_SearchPath == NULL) { + PyErr_SetString(PyExc_SystemError, + "codec module not properly initialized"); + goto onError; + } if (!import_encodings_called) import_encodings(); @@ -109,6 +114,7 @@ result = PyDict_GetItem(_PyCodec_SearchCache, v); if (result != NULL) { Py_INCREF(result); + Py_DECREF(v); return result; } @@ -121,6 +127,7 @@ if (args == NULL) goto onError; PyTuple_SET_ITEM(args,0,v); + v = NULL; for (i = 0; i < len; i++) { PyObject *func; @@ -146,7 +153,7 @@ if (i == len) { /* XXX Perhaps we should cache misses too ? */ PyErr_SetString(PyExc_LookupError, - "unkown encoding"); + "unknown encoding"); goto onError; } @@ -156,6 +163,7 @@ return result; onError: + Py_XDECREF(v); Py_XDECREF(args); return NULL; } @@ -378,5 +386,7 @@ void _PyCodecRegistry_Fini() { Py_XDECREF(_PyCodec_SearchPath); + _PyCodec_SearchPath = NULL; Py_XDECREF(_PyCodec_SearchCache); + _PyCodec_SearchCache = NULL; }

2 1

Unicode patches checked in
by Guido van Rossum March 16, 2000

March 16, 2000

I've just checked in a massive patch from Marc-Andre Lemburg which adds Unicode support to Python. This work was financially supported by Hewlett-Packard. Marc-Andre has done a tremendous amount of work, for which I cannot thank him enough. We're still awaiting some more things: Marc-Andre gave me documentation patches which will be reviewed by Fred Drake before they are checked in; Fredrik Lundh has developed a new regular expression which is Unicode-aware and which should be checked in real soon now. Also, the documentation is probably incomplete and will be updated, and of course there may be bugs -- this should be considered alpha software. However, I believe it is quite good already, otherwise I wouldn't have checked it in! I'd like to invite everyone with an interest in Unicode or Python 1.6 to check out this new Unicode-aware Python, so that we can ensure a robust code base by the time Python 1.6 is released (planned release date: June 1, 2000). The download links are below. Links: http://www.python.org/download/cvs.html Instructions on how to get access to the CVS version. (David Ascher is making nightly tarballs of the CVS version available at http://starship.python.net/crew/da/pythondists/) http://starship.python.net/crew/lemburg/unicode-proposal.txt The latest version of the specification on which the Marc has based his implementation. http://www.python.org/sigs/i18n-sig/ Home page of the i18n-sig (Internationalization SIG), which has lots of other links about this and related issues. http://www.python.org/search/search_bugs.html The Python Bugs List. Use this for all bug reports. Note that next Tuesday I'm going on a 10-day trip, with limited time to read email and no time to solve problems. The usual crowd will take care of urgent updates. See you at the Intel Computing Continuum Conference in San Francisco or at the Python Track at Software Development 2000 in San Jose! --Guido van Rossum (home page: http://www.python.org/~guido/)

11 26

Finalizers considered questionable ;)
by Jim Fulton March 16, 2000

March 16, 2000

Here's my $0.02. I agree with the sentiments that use of finalizers should be discouraged. They are extremely helpful in cases like tempfile.TemporaryFileWrapper, so I think that they should be supported. I do think that the language should not promise a high level of service. Some observations: - I spent a little bit of time on the ANSI Smalltalk committee, where I naively advocated adding finalizers to the language. I was resoundingly told no. :) - Most of the Python objects I deal with these days are persistent. Their lifetimes are a lot more complicated that most Python objects. They get created once, but they get loaded into and out of memory many times. In fact, they can be in memory many times simultaneously. :) A couple of years ago I realized that it only made sense to call __init__ when an object was first created, not when it is subsequently (re)loaded into memory. This led to a change in Python pickling semantics and the deprecation of the loathsome __getinitargs__ protocol. :) For me, a similar case can be made against use of __del__ for persistent objects. For persistent objects, a __del__ method should only be used for cleaning up the most volatile of resources. A persistent object __del__ should not perform any semantically meaningful operations because __del__ has no semantic meaning. - Zope has a few uses of __del__. These are all for non-persistent objects. Interesting, in grepping for __del__, I found a lot of cases where __del__ was used and then commented out. Finalizers seem to be the sort of thing that people want initially and then get over. I'm inclined to essentially keep the current rules and simply not promise that __del__ will be able to run correctly. That is, Python should call __del__ and ignore exceptions raised (or provide some *optional* logging or other debugging facility). There is no reason for __del__ to fail unless it depends on cyclicly-related objects, which should be viewed as a design mistake. OTOH, __del__ should never fail because module globals go away. IMO, the current circular references involving module globals are unnecessary, but that's a different topic. ;) Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

3 2

Allowing multiple socket maps in asyncore (and asynchat)
by Jim Fulton March 15, 2000

March 15, 2000

I find asyncore to be quite useful, however, it is currently geared to having a single main loop. It uses a global socket map that all asyncore dispatchers register with. I have an application in which I want to have multiple socket maps. I propose that we start moving toward a model in which selection of a socket map and control of the asyncore loop is a bit more explicit. If no one objects, I'll work up some initial patches. Who should I submit these to? Sam? Should the medusa public CVS form the basis? Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

2 1

Python 1.7 tokenization feature request
by gvwilson＠nevex.com March 14, 2000

March 14, 2000

Once 1.6 is out the door, would people be willing to consider extending Python's token set to make HTML/XML-ish spellings using entity references legal? This would make the following 100% legal Python: i = 0 while i < 10: print i & 1 i = i + 1 which would in turn make it easier to embed Python in XML such as config-files-for-whatever-Software-Carpentry-produces-to-replace-make, PMZ, and so on. Greg

10 17

unicode objects and C++
by Mark Hammond March 14, 2000

March 14, 2000

I struck a bit of a snag with the Unicode support when trying to use the most recent update in a C++ source file. The problem turned out to be that unicodeobject.h did a #include "wchar.h", but did it while an 'extern "C"' block was open. This upset the MSVC6 wchar.h, as it has special C++ support. Attached below is a patch I made to unicodeobject.h that solved my problem and allowed my compilations to succeed. Theoretically the same problem could exist for wctype.h, and probably lots of other headers, but this is the immediate problem :-) An alternative patch would be to #include "whcar.h" in PC\config.h outside of any 'extern "C"' blocks - wchar.h on Windows has guards that allows for multiple includes, so the unicodeobject.h include of that file will succeed, but not have the side-effect it has now. Im not sure what the preferred solution is - quite possibly the PC\config.h change, but Ive include the unicodeobject.h patch anyway :-) Mark. *** unicodeobject.h 2000/03/13 23:22:24 2.2 --- unicodeobject.h 2000/03/14 01:06:57 *************** *** 85,91 **** --- 85,101 ---- #endif #ifdef HAVE_WCHAR_H + + #ifdef __cplusplus + } /* Close the 'extern "C"' before bringing in system headers */ + #endif + # include "wchar.h" + + #ifdef __cplusplus + extern "C" { + #endif + #endif #ifdef HAVE_USABLE_WCHAR_T

2 1

FW: Fixing os.popen on Win32 => is the win32pipe stuff going to be adopted?
by Tim Peters March 13, 2000

March 13, 2000

Mike has a darned good point here. Anyone have a darned good answer <wink>? -----Original Message----- From: python-list-admin(a)python.org [mailto:python-list-admin@python.org] On Behalf Of Mike Fletcher Sent: Tuesday, March 07, 2000 2:08 PM To: Python Listserv (E-mail) Subject: Fixing os.popen on Win32 => is the win32pipe stuff going to be adopted? Just reading one more post (and a FAQ) on the win32 pipe breakage (sewage all over the hard-disk, traffic rerouted through the bit-bucket, you aren't getting to work anytime soon Mrs. Programmer) and wondering why we have a FAQ instead of having the win32pipe stuff rolled into the os module to fix it. Is there some incompatibility? Is there a licensing problem? Ideas? Mike __________________________________ Mike C. Fletcher Designer, VR Plumber http://members.home.com/mcfletch -- http://www.python.org/mailman/listinfo/python-list

10 30

string.replace behaviour change since Unicode patch.
by Mark Hammond March 13, 2000

March 13, 2000

Hi, After applying the Unicode changes string.replace() seems to have changed its behaviour: Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import string >>> string.replace("foo\nbar", "\n", "") 'foobar' >>> But since the Unicode update: Python 1.5.2+ (#0, Feb 2 2000, 16:46:55) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import string >>> string.replace("foo\nbar", "\n", "") Traceback (innermost last): File "<stdin>", line 1, in ? File "L:\src\python-cvs\lib\string.py", line 407, in replace return s.replace(old, new, maxsplit) ValueError: empty replacement string >>> The offending check is stringmodule.c, line 1578: if (repl_len <= 0) { PyErr_SetString(PyExc_ValueError, "empty replacement string"); return NULL; } Changing the check to "< 0" fixes the immediate problem, but it is unclear why the check was added at all, so I didnt bother submitting a patch... Mark.

2 1

Py3K: indirect coupling between raise and exception handler
by Skip Montanaro March 11, 2000

March 11, 2000

Consider the following snippet of code from MySQLdb.py: try: self._query(query % escape_row(args, qc)) except TypeError: self._query(query % escape_dict(args, qc)) It's not quite right. There are at least four reasons I can think of why the % operator might raise a TypeError: 1. query has not enough format specifiers 2. query has too many format specifiers 3. argument type mismatch between individual format specifier and corresponding argument 4. query expects dist-style interpolation The except clause only handles the last case. That leaves the other three cases mishandled. The above construct pretends that all TypeErrors possible are handled by calling escape_dict() instead of escape_row(). I stumbled on case 2 yesterday and got a fairly useless error message when the code in the except clause also bombed. Took me a few minutes of head scratching to see that I had an extra %s in my format string. A note to Andy Dustman, MySQLdb's author, yielded the following modified version: try: self._query(query % escape_row(args, qc)) except TypeError, m: if m.args[0] == "not enough arguments for format string": raise if m.args[0] == "not all arguments converted": raise self._query(query % escape_dict(args, qc)) This will do the trick for me for the time being. Note, however, that the only way for Andy to decide which of the cases occurred (case 3 still isn't handled above, but should occur very rarely in MySQLdb since it only uses the more accommodating %s as a format specifier) is to compare the string value of the message to see which of the four cases was raised. This strong coupling via the error message text between the exception being raised (in C code, in this case) and the place where it's caught seems bad to me and encourages authors to either not recover from errors or to recover from them in the crudest fashion. If Guido decides to tweak the TypeError message in any fashion, perhaps to include the count of arguments in the format string and argument tuple, this code will break. It makes me wonder if there's not a better mechanism waiting to be discovered. Would it be possible to publish an interface of some sort via the exceptions module that would allow symbolic names or dictionary references to be used to decide which case is being handled? I envision something like the following in exceptions.py: UNKNOWN_ERROR_CATEGORY = 0 TYP_SHORT_FORMAT = 1 TYP_LONG_FORMAT = 2 ... IND_BAD_RANGE = 1 message_map = { # leave (TypeError, ("not enough arguments for format string",)): TYP_SHORT_FORMAT, (TypeError, ("not all arguments converted",)): TYP_LONG_FORMAT, ... (IndexError, ("list index out of range",)): IND_BAD_RANGE, ... } This would isolate the raw text of exception strings to just a single place (well, just one place on the exception handling side of things). It would be used something like try: self._query(query % escape_row(args, qc)) except TypeError, m: from exceptions import * exc_case = message_map.get((TypeError, m.args), UNKNOWN_ERROR_CATEGORY) if exc_case in [UNKNOWN_ERROR_CATEGORY,TYP_SHORT_FORMAT, TYP_LONG_FORMAT]: raise self._query(query % escape_dict(args, qc)) This could be added to exceptions.py without breaking existing code. Does this (or something like it) seem like a reasonable enhancement for Py2K? If we can narrow things down to an implementable solution I'll create a patch. Skip Montanaro | http://www.mojam.com/ skip(a)mojam.com | http://www.musi-cal.com/

4 5