Mailman 3 August 2006 - Python-Dev

SimpleXMLWriter missing from elementtree
by Ralf Schmitt Aug. 14, 2006

Aug. 14, 2006

Hi, any chance to get SimpleXMLWriter (and other modules maybe??) getting included into xml.etree? Otherwise people might have to stick to the original elementtree, which doesn't really make sense, since most of elementtree already is included. - Ralf

3 2

Dict suppressing exceptions
by Guido van Rossum Aug. 14, 2006

Aug. 14, 2006

I've been happily ignoring python-dev for the last three weeks or so, and Neal just pointed me to some thorny issues that are close to resolution but not quite yet resolved, yet need to be before beta 3 on August 18 (Friday next week). Here's my take on the dict-suppressing-exceptions issue (I'll send out separate messages for each issue where Neal has asked me to weigh in). It wasn't my idea to stop ignoring exceptions in dict lookups; I would gladly have put this off until Py3k, where the main problem (str-unicode __eq__ raising UnicodeError) will go away. But since people are adamant that they want this in sooner, I suggest that to minimize breakage we could make an exception for str-unicode comparisons. I came up with the following code to reproduce the issue; this prints 0 in 2.2, False in 2.3 and 2.4, but raises UnicodeDecodeError in 2.5 (head): a = {u"1": 1} x = hash(u"1") class C(str): def __hash__(s): return x print C("\xff") in a The following patch makes this print False in 2.5 again. Notes about the patch: - this also fixes an out-of-date comment that should be fixed even if the rest of the idea is rejected (lookdict_string() can return NULL when it calls lookdict) - the exception could be narrowed even further by only suppressing the exception when startkey and key are both either str or unicode instances. What do people think? --- Objects/dictobject.c (revision 51180) +++ Objects/dictobject.c (working copy) @@ -230,7 +230,8 @@ lookdict() is general-purpose, and may return NULL if (and only if) a comparison raises an exception (this was new in Python 2.5). lookdict_string() below is specialized to string keys, comparison of which can -never raise an exception; that function can never return NULL. For both, when +never raise an exception; that function can never return NULL (except when it +decides to replace itself with the more general lookdict()). For both, when the key isn't found a dictentry* is returned for which the me_value field is NULL; this is the slot in the dict at which the key would have been found, and the caller can (if it wishes) add the <key, value> pair to the returned @@ -259,8 +260,13 @@ if (ep->me_hash == hash) { startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); - if (cmp < 0) - return NULL; + if (cmp < 0) { + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { + PyErr_Clear(); + } + else + return NULL; + } if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) return ep; @@ -289,8 +295,13 @@ if (ep->me_hash == hash && ep->me_key != dummy) { startkey = ep->me_key; cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); - if (cmp < 0) - return NULL; + if (cmp < 0) { + if (PyErr_Occurred()==PyExc_UnicodeDecodeError) { + PyErr_Clear(); + } + else + return NULL; + } if (ep0 == mp->ma_table && ep->me_key == startkey) { if (cmp > 0) return ep; -- --Guido van Rossum (home page: http://www.python.org/~guido/)

6 19

2.6 idea: a 'function' builtin to parallel classmethod and staticmethod
by Nick Coghlan Aug. 14, 2006

Aug. 14, 2006

It's sometimes useful to be able to use an existing callable as a method of a new class. If the callable is a real function, this is easy. You just including the following line in the class definition: method = some_callable However, callable objects without a function-like __get__ method can't be used that way. So, to avoid a dependency on an implementation detail of some_callable (i.e. whether or not it is a true function object), you have to write: def method(): return some_callable() (and you can lose useful metadata in the process!) However, if you're adding a callable as a class method or static method, there is OOWTDI: method = classmethod(some_callable) method = staticmethod(some_callable) It would be nice if there was a similar mechanism for normal instance methods as well: method = function(some_callable) This came up during the PEP 343 implementation - "context = contextlib.closing" is a tempting thing to write in order to provide a "x.context()" method for use in a with statement, but it doesn't actually work properly (because closing is a class, not a function). Similarly, you can't create a method simply by applying functools.partial to an existing function - the result won't have a __get__ method, so it will be treated like a normal attribute instead of as an instance method. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

4 4

dict containment annoyance
by tomer filiba Aug. 14, 2006

Aug. 14, 2006

[Aahz] > -1 > > This is seriously no different from an attempt to do > > >>> a = {} > >>> a[ [] ] = 1 how so? i'm not trying to modify/store anything in a dict. i'm only testing if a certain object is contained in the dict. that's totally different. imagine this: >>> x = 6 >>> x < 2 am i trying to modify 'x' in any way? i'm only testing if x is less than 2. i fail to see your point [David Hopwood] > However, if it isn't hashable, asking whether it is in a dict is very > likely to be a programming error. not really. my program holds a blacklist of "bad objects", which shouldn't be granted access to. instead of a list, i'm using a set (performance, whatever). the "bad objects" are all hashable, and i want to see if an arbitrary object is "good" or "bad", by looking it up in the blacklist. the arbitrary object can be hashable or not, that's not the question. just like `"hello" != 2` returns True, instead of raising IncompatibleTypes, i want `x in y` to return False when `x` is not contained in `y` (with no regard to whether or not it's possible for `y` to contain `x`). the logic is simple: every `x` is either contained in `y` or not. if `x` *cannot* be contained in `y`, then the answer is a "strong no", but that's still a "no". -tomer

6 6

test_socketserver failure on cygwin
by Neal Norwitz Aug. 14, 2006

Aug. 14, 2006

I'm not sure if this is a race condition in the test, the test using SocketServer inappropriately, or if the failure is exposing a problem in SocketServer. The issue is that when the second (UDP) server is setup, there is still a child process from the first (TCP) server. So when the UDP server calls collect_children(), it gets back the pid from the TCP server, which is not in the UDP server's list and an exception is raised. In general, this is a problem, if program is using a ForkingServer and spawns it's own children, this problem can occur. This is because os.waitpid() is called with a pid of 0, rather than the individual pids in the list. I suppose that would fix the problem. Is this worth fixing? (It's a pain to handle properly because there is also use of a blocking wait.) http://www.python.org/dev/buildbot/all/x86%20cygwin%20trunk/builds/1107/ste… Add this patch corrects the problem (see below for more details): Index: Lib/test/test_socketserver.py =================================================================== --- Lib/test/test_socketserver.py (revision 51260) +++ Lib/test/test_socketserver.py (working copy) @@ -183,7 +183,11 @@ def testall(): testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream) + time.sleep(.1) + reap_children() testloop(socket.AF_INET, udpservers, MyDatagramHandler, testdgram) + time.sleep(.1) + reap_children() if hasattr(socket, 'AF_UNIX'): testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream) # Alas, on Linux (at least) recvfrom() doesn't return a meaningful So modifying SocketServer like this: -bash-3.1$ svn diff Lib/SocketServer.py Lib/test/test_socketserver.py Index: Lib/SocketServer.py =================================================================== --- Lib/SocketServer.py (revision 51260) +++ Lib/SocketServer.py (working copy) @@ -420,6 +420,7 @@ except os.error: pid = None if not pid: break + print >> sys.__stdout__, 'pid', pid, 'active', self.active_children self.active_children.remove(pid) def process_request(self, request, client_address): @@ -428,6 +429,7 @@ pid = os.fork() if pid: # Parent process + print >> sys.__stdout__, 'in parent, adding', pid, self.active_children if self.active_children is None: self.active_children = [] self.active_children.append(pid) We see this on error (prior to the test patch above): test_socketserver in parent, adding 3084 None pid 3084 active [3084] in parent, adding 2408 [] pid 2408 active [2408] in parent, adding 3908 [] in parent, adding 532 None pid 3908 active [532] Exception in thread Thread-12: Traceback (most recent call last): ... File "/home/neal/build/python/trunk/Lib/SocketServer.py", line 424, in collect_children self.active_children.remove(pid) ValueError: list.remove(x): x not in list

1 0

What is the status of file.readinto?
by Alexander Belopolsky Aug. 13, 2006

Aug. 13, 2006

I know that it is a subject of an annual discussion, but since I could not find any mention of it in the last year, let me ask this question again: why is file.readinto "Undocumented"? It is quite useful, particularly with numpy. Would a patch adding "readinto" to StringIO be acceptable?

3 3

Re: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/class
by Tim Peters Aug. 13, 2006

Aug. 13, 2006

[Travis E. Oliphant] > Right now throughout the Python code it is assumed that > sizeof(Py_ssize_t) <= sizeof(long). If you find any code like that, it's a bug. Win64 is a platform where it's false.

2 1

dict containment annoyance
by tomer filiba Aug. 12, 2006

Aug. 12, 2006

>>> a={1:2, 3:4} >>> [] in a Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: list objects are unhashable >>> imo, the expression should just evaluate to False instead of raising an exception. it's a question of semantics -- i asked whether the object (a list, in this case) is contained in the dict. i didn't ask whether it's hashable or not. if the object is unhashable, then surely it's not contained in the dict, therefore the answer is False, rather than an exception. dict.__contains__ should just swallow these exceptions, and return False. -tomer

3 2

[Python-3000] Python 2.5 release schedule (was: threading, part 2)
by Aahz Aug. 12, 2006

Aug. 12, 2006

[added python-dev to make sure everyone sees this] On Sat, Aug 12, 2006, Josiah Carlson wrote: > > All other feature additions are too late in the Beta cycle (Beta 3 is > next week) For some reason, this is the second time I've seen this claim. Beta 3 was released August 3 and next week is rc1. We are right now in complete feature lockdown; even documenting an existing API IMO requires approval from the Release Manager. -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan

1 0

Recent logging spew
by Tim Peters Aug. 12, 2006

Aug. 12, 2006

We're getting an awful lot of these turds at the ends of test runs now: Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 1351, in shutdown h.flush() File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file Error in sys.exitfunc: Traceback (most recent call last): File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 1351, in shutdown h.flush() File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py", line 731, in flush self.stream.flush() ValueError: I/O operation on closed file [450168 refs] program finished with exit code 0 Not sure what caused it, but suspect this: """ Author: georg.brandl Date: Fri Aug 11 09:26:10 2006 New Revision: 51206 Modified: python/trunk/Lib/logging/__init__.py python/trunk/Misc/NEWS Log: logging's atexit hook now runs even if the rest of the module has already been cleaned up. """ Anyway, can we stop this? It doesn't look like "a feature" from here ;-)

2 1