Mailman 3 March 2002 - Python-Dev

Weird new test failures: import test.XYZ failing
by Tim Peters March 10, 2002

March 10, 2002

I'm seeing three seemingly unrelated failures on Windows, seemingly unrelated to each other or to anything that's been checked in since the last time the tests passed. Anyone else? I blew away my intermediate files and rebuilt Python from scratch without visible change; debug-mode builds act the same too; only test_pyclbr.py was changed recently, but in a way that doesn't relate to the failure mode. Hmm -- but every one of them *is* trying to import something with a name of the form "test.XYZ". Ring a bell to anyone? C:\Code\python\PCbuild>python ../lib/test/test_pyclbr.py test_easy (__main__.PyclbrTest) ... ok test_others (__main__.PyclbrTest) ... ERROR ====================================================================== ERROR: test_others (__main__.PyclbrTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "../lib/test/test_pyclbr.py", line 153, in test_others module=sys.modules[__name__]) File "../lib/test/test_pyclbr.py", line 60, in checkModule dict = pyclbr.readmodule_ex(moduleName) File "C:\CODE\PYTHON\lib\pyclbr.py", line 171, in readmodule_ex child = readmodule_ex(submodule, parent['__path__'], 1) KeyError: __path__ ---------------------------------------------------------------------- Ran 2 tests in 2.090s FAILED (errors=1) Traceback (most recent call last): File "../lib/test/test_pyclbr.py", line 167, in ? test_main() File "../lib/test/test_pyclbr.py", line 163, in test_main run_unittest(PyclbrTest) File "../lib/test\test_support.py", line 190, in run_unittest run_suite(unittest.makeSuite(testclass), testclass) File "../lib/test\test_support.py", line 185, in run_suite raise TestFailed(err) test_support.TestFailed: Traceback (most recent call last): File "../lib/test/test_pyclbr.py", line 153, in test_others module=sys.modules[__name__]) File "../lib/test/test_pyclbr.py", line 60, in checkModule dict = pyclbr.readmodule_ex(moduleName) File "C:\CODE\PYTHON\lib\pyclbr.py", line 171, in readmodule_ex child = readmodule_ex(submodule, parent['__path__'], 1) KeyError: __path__ The full line there is cm('test.test_pyclbr', module=sys.modules[__name__]) C:\Code\python\PCbuild>python ../lib/test/test_descrtut.py Traceback (most recent call last): File "../lib/test/test_descrtut.py", line 501, in ? test_main(1) File "../lib/test/test_descrtut.py", line 496, in test_main import test_support, test.test_descrtut ImportError: No module named test_descrtut An obvious attempt to import test.test_descrtut. C:\Code\python\PCbuild>python ../lib/test/test_charmapcodec.py Traceback (most recent call last): File "../lib/test/test_charmapcodec.py", line 21, in ? check(unicode('abc', codecname), u'abc') LookupError: unknown encoding At line 21, codecname is codecname = 'test.testcodec'

1 1

Please review PEP 278 - Universal newline support
by Jack Jansen March 10, 2002

March 10, 2002

Folks, PEP 278 has been quietly sitting there with nothing much happening, after some initial discussion with two or three people. First question: would people please review it, and preferrably also test it (esp. windows and linux are platforms on which I would like to see it tested). Second question: what happens next? How is the PEP accepted? -- - Jack Jansen <Jack.Jansen(a)oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

7 7

Yet another (?) for-with-indices
by Armin Rigo March 9, 2002

March 9, 2002

Hello everybody, Here is yet another proposal (or did I miss something similar?) about the for-with-explicit-indices syntax problems. Now that Python has iterators, they could be more explicitely used by user code. More specifically, as it seems we often loop over sequence elements and occasionnally need an explicit index, I suggest adding a 'count' attribute to sequence iterators that returns the number of items returned so far, that is, the index of the *next* item (as stored in the iterator in C): it = iter(lst) for x in it: if need_the_index_of_x: index = it.count-1 For example, to decrement all positive integers in a list, modifying it in-place: it = iter(lst) for x in it: if x>0: lst[it.count-1] -= 1 No syntax sugar is required. All the trick is in explicitely naming the iterator object used by the loop. This idea could maybe be extended to make the attribute writeable. For example, inside a loop: it.count += 1 # skip next element it.count -= 1 # process the same element again it.count = 0 # full rewind it.count -= 1; del lst[it.count] # remove bad item from list Thanks, Armin.

1 0

�� �� ִϸ��̼� ��Ʈ(��.��)
by �� March 9, 2002

March 9, 2002

1 0

Re: [Python-Dev] Moving bugs and patches through the pipeline more quickly
by Michael Hudson March 9, 2002

March 9, 2002

Tim Peters <tim.one(a)comcast.net> writes: > [Jeremy Hylton] > > When we were working on Python 2.0, PythonLabs made a > > serious commitment to keep the list of bugs on one page. > > Lots of people fixed bugs to achieve that goal, and more > > processing power will definitely help. > > Note that we had full-time jobs working on Python then too. Well, not > entirely: at the end of the BeOpen run, all of PythonLabs was unemployed, > so we got to spend 1200% of every day volunteering to finish 2.0. I think there are more bugs being submitted now, too. We should never have told anyone about that tracker <wink>. > > One other thing that helped was that I spent many hours each > > week tracking bugs and making sure someone was working on > > them. I intend to pick that task up again for Python 2.3. > > It would be great if there were more developers to lean on > > for the bugs. > > During the times I did that task, I spent about 30 hours per week on bug + > patch triage alone. > > It would be hard to overestimate how much concerted effort it would > take to get back to "one page" again; the SF stats (I think only > admins can view the reports) Nope. At least, I can see them. > show that we're falling further behind month by month. The > "Feature Requests" tracker may as well be a trash can. It's probably better that PEP 42. That should probably be sliced up and moved back into the tracker. > OTOH, we could make a lot of progress very quickly by agreeing to > drop Python support for all save the OS + compiler Guido happens to > use <wink>. Certainly. Life would be easier if we didn't have to worry about bugs like: [ 459464 ] Math_test overflowerror on sparc64 linux (to pick a random example). I don't think keeping open bugs <50 is a realistic goal unless there's at least one person working full time on keeping things that way, and that doesn't seem likely. OTOH, a certain amount of progress is being made as the result of the current guilt trip -- let's see how long that lasts <wink>. Cheers, M. -- This is an off-the-top-of-the-head-and-not-quite-sober suggestion, so is probably technically laughable. I'll see how embarassed I feel tomorrow morning. -- Patrick Gosling, ucam.comp.misc

5 5

RE: [Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?
by Tim Peters March 9, 2002

March 9, 2002

I hope the bogus underflow problem is fixed in CVS Python now. Since my platform didn't have the problem (and still doesn't, of course), we won't know for sure until people try it and report back. Python used to link with -lieee, but somebody changed it for a reason history may have lost. See comments attached to Huaiyu Zhu's bug report for more on that: <http://sf.net/tracker/?func=detail&aid=525705&group_id=5470&atid=105470> If someone on a box that had the problem can build current CVS Python with and without -lieee (I'm told it defaults to "without" today, although that appears to mixed up with whether or not __fpu_control is found outside of libieee (search for "ieee" in configure.in)), please try the following in a shell: import math x = 1e200 y = 1/x math.pow(x, 2) # expect OverflowError math.pow(y, 2) # expect 0. pow(x, 2) # expect OverflowError pow(y, 2) # expect 0. x**2 # expect OverflowError y**2 # expect 0. If those all work as annotated on a box that had the problem, please report the success *as a comment to the bug report* (link above). If one or more still fail, likewise please give box details and paste the failures into a comment on the bug report. Thanks!

1 0

PEP 282: A Logging System -- comments please
by Trent Mick March 9, 2002

March 9, 2002

Howdy all, I would appreciate any comments you might have on this proposal for adding a logging system to the Python Standard Library. This PEP is still an early draft so please forward your comments just to me directly for now. Thanks, Trent ----------------------------------------------------------- PEP: 282 Title: A Logging System Version: $Revision: 1.1 $ Last-Modified: $Date: 2002/02/15 04:09:17 $ Author: trentm(a)activestate.com (Trent Mick) Status: Draft Type: Standards Track Created: 4-Feb-2002 Python-Version: 2.3 Post-History: Abstract This PEP describes a proposed logging package for Python's standard library. Basically the system involves the user creating one or more logging objects on which methods are called to log debugging notes/general information/warnings/errors/etc. Different logging 'levels' can be used to distinguish important messages from trivial ones. A registry of named singleton logger objects is maintained so that 1) different logical logging streams (or 'channels') exist (say, one for 'zope.zodb' stuff and another for 'mywebsite'-specific stuff) 2) one does not have to pass logger object references around. The system is configurable at runtime. This configuration mechanism allows one to tune the level and type of logging done while not touching the application itself. Motivation If a single logging mechanism is enshrined in the standard library, 1) logging is more likely to be done 'well', and 2) multiple libraries will be able to be integrated into larger applications which can be logged reasonably coherently. Influences This proposal was put together after having somewhat studied the following logging packages: o java.util.logging in JDK 1.4 (a.k.a. JSR047) [1] o log4j [2] These two systems are *very* similar. o the Syslog package from the Protomatter project [3] o MAL's mx.Log package [4] This proposal will basically look like java.util.logging with a smattering of log4j. Simple Example This shows a very simple example of how the logging package can be used to generate simple logging output on stdout. --------- mymodule.py ------------------------------- import logging log = logging.getLogger("MyModule") def doit(): log.debug("doin' stuff") # do stuff ... ----------------------------------------------------- --------- myapp.py ---------------------------------- import mymodule, logging log = logging.getLogger("MyApp") log.info("start my app") try: mymodule.doit() except Exception, e: log.error("There was a problem doin' stuff.") log.info("end my app") ----------------------------------------------------- > python myapp.py 0 [myapp.py:4] INFO MyApp - start my app 36 [mymodule.py:5] DEBUG MyModule - doin' stuff 51 [myapp.py:9] INFO MyApp - end my app ^^ ^^^^^^^^^^^^ ^^^^ ^^^^^ ^^^^^^^^^^ | | | | `-- message | | | `-- logging name/channel | | `-- level | `-- location `-- time NOTE: Not sure exactly what the default format will look like yet. Control Flow [Note: excerpts from Java Logging Overview. [5]] Applications make logging calls on *Logger* objects. Loggers are organized in a hierarchical namespace and child Loggers may inherit some logging properties from their parents in the namespace. Notes on namespace: Logger names fit into a "dotted name" namespace, with dots (periods) indicating sub-namespaces. The namespace of logger objects therefore corresponds to a single tree data structure. "" is the root of the namespace "Zope" would be a child node of the root "Zope.ZODB" would be a child node of "Zope" These Logger objects allocate *LogRecord* objects which are passed to *Handler* objects for publication. Both Loggers and Handlers may use logging *levels* and (optionally) *Filters* to decide if they are interested in a particular LogRecord. When it is necessary to publish a LogRecord externally, a Handler can (optionally) use a *Formatter* to localize and format the message before publishing it to an I/O stream. Each Logger keeps track of a set of output Handlers. By default all Loggers also send their output to their parent Logger. But Loggers may also be configured to ignore Handlers higher up the tree. The APIs are structured so that calls on the Logger APIs can be cheap when logging is disabled. If logging is disabled for a given log level, then the Logger can make a cheap comparison test and return. If logging is enabled for a given log level, the Logger is still careful to minimize costs before passing the LogRecord into the Handlers. In particular, localization and formatting (which are relatively expensive) are deferred until the Handler requests them. Levels The logging levels, in increasing order of importance, are: DEBUG INFO WARN ERROR FATAL ALL This is consistent with log4j and Protomatter's Syslog and not with JSR047 which has a few more levels and some different names. Implementation-wise: these are just integer constants, to allow simple comparison of importance. See "What Logging Levels?" below for a debate on what standard levels should be defined. Loggers Each Logger object keeps track of a log level (or threshold) that it is interested in, and discards log requests below that level. The *LogManager* maintains a hierarchical namespace of named Logger objects. Generations are denoted with dot-separated names: Logger "foo" is the parent of Loggers "foo.bar" and "foo.baz". The main logging method is: class Logger: def log(self, level, msg, *args): """Log 'msg % args' at logging level 'level'.""" ... however convenience functions are defined for each logging level: def debug(self, msg, *args): ... def info(self, msg, *args): ... def warn(self, msg, *args): ... def error(self, msg, *args): ... def fatal(self, msg, *args): ... XXX How to defined a nice convenience function for logging an exception? mx.Log has something like this, doesn't it? XXX What about a .raising() convenience function? How about: def raising(self, exception, level=ERROR): ... It would create a log message describing an exception that is about to be raised. I don't like that 'level' is not first when it *is* first for .log(). Handlers Handlers are responsible for doing something useful with a given LogRecord. The following core Handlers will be implemented: - StreamHandler: A handler for writing to a file-like object. - FileHandler: A handler for writing to a single file or set of rotating files. More standard Handlers may be implemented if deemed desirable and feasible. Other interesting candidates: - SocketHandler: A handler for writing to remote TCP ports. - CreosoteHandler: A handler for writing to UDP packets, for low-cost logging. Jeff Bauer already had such a system [5]. - MemoryHandler: A handler that buffers log records in memory (JSR047). - SMTPHandler: Akin to log4j's SMTPAppender. - SyslogHandler: Akin to log4j's SyslogAppender. - NTEventLogHandler: Akin to log4j's NTEventLogAppender. Formatters A Formatter is responsible for converting a LogRecord to a string representation. A Handler may call its Formatter before writing a record. The following core Formatters will be implemented: - Formatter: Provide printf-like formatting, perhaps akin to log4j's PatternAppender. Other possible candidates for implementation: - XMLFormatter: Serialize a LogRecord according to a specific schema. Could copy the schema from JSR047's XMLFormatter or log4j's XMLAppender. - HTMLFormatter: Provide a simple HTML output of log information. (See log4j's HTMLAppender.) Filters A Filter can be called by a Logger or Handler to decide if a LogRecord should be logged. JSR047 and log4j have slightly different filtering interfaces. The former is simpler: class Filter: def isLoggable(self): """Return a boolean.""" The latter is modeled after Linux's ipchains (where Filter's can be chained with each filter either 'DENY'ing, 'ACCEPT'ing, or being 'NEUTRAL' on each check). I would probably favor to former because it is simpler and I don't immediate see the need for the latter. No filter implementations are currently proposed (other that the do nothing base class) because I don't have enough experience to know what kinds of filters would be common. Users can always subclass Filter for their own purposes. Log4j includes a few filters that might be interesting. Configuration Note: Configuration for the proposed logging system is currently under-specified. The main benefit of a logging system like this is that one can control how much and what logging output one gets from an application without changing that application's source code. Log4j and Syslog provide for configuration via an external XML file. Log4j and JSR047 provide for configuration via Java properties (similar to -D #define's to a C/C++ compiler). All three provide for configuration via API calls. Configuration includes the following: - What logging level a logger should be interested in. - What handlers should be attached to which loggers. - What filters should be attached to which handlers and loggers. - Specifying attributes specific to certain Handlers and Filters. - Defining the default configuration. - XXX Add others. In general each application will have its own requirements for how a user may configure logging output. One application (e.g. distutils) may want to control logging levels via '-q,--quiet,-v,--verbose' options to setup.py. Zope may want to configure logging via certain environment variables (e.g. 'STUPID_LOG_FILE' :). Komodo may want to configure logging via its preferences system. This PEP proposes to clearly document the API for configuring each of the above listed configurable elements and to define a reasonable default configuration. This PEP does not propose to define a general XML or .ini file configuration schema and the backend to parse it. It might, however, be worthwhile to define an abstraction of the configuration API to allow the expressiveness of Syslog configuration. Greg Wilson made this argument: In Protomatter [Syslog], you configure by saying "give me everything that matches these channel+level combinations", such as "server.error" and "database.*". The log4j "configure by inheritance" model, on the other hand, is very clever, but hard for non-programmers to manage without a GUI that essentially reduces it to Protomatter's. Case Scenarios This section presents a few usage scenarios which will be used to help decide how best to specify the logging API. (1) A short simple script. This script does not have many lines. It does not heavily use any third party modules (i.e. the only code doing any logging would be the main script). Only one logging channel is really needed and thus, the channel name is unnecessary. The user doesn't want to bother with logging system configuration much. (2) Medium sized app with C extension module. Includes a few Python modules and a main script. Employs, perhaps, a few logging channels. Includes a C extension module which might want to make logging calls as well. (3) Distutils. A large number of Python packages/modules. Perhaps (but not necessarily) a number of logging channels are used. Specifically needs to facilitate the controlling verbosity levels via simple command line options to 'setup.py'. (4) Large, possibly multi-language, app. E.g. Zope or (my experience) Komodo. (I don't expect this logging system to deal with any cross-language issues but it is something to think about.) Many channels are used. Many developers involved. People providing user support are possibly not the same people who developed the application. Users should be able to generate log files (i.e. configure logging) while reproducing a bug to send back to developers. Implementation XXX Details to follow consensus that this proposal is a good idea. What Logging Levels? The following are the logging levels defined by the systems I looked at: - log4j: DEBUG, INFO, WARN, ERROR, FATAL - syslog: DEBUG, INFO, WARNING, ERROR, FATAL - JSR047: FINEST, FINER, FINE, CONFIG, INFO, WARNING, SEVERE - zLOG (used by Zope): TRACE=-300 -- Trace messages DEBUG=-200 -- Debugging messages BLATHER=-100 -- Somebody shut this app up. INFO=0 -- For things like startup and shutdown. PROBLEM=100 -- This isn't causing any immediate problems, but deserves attention. WARNING=100 -- A wishy-washy alias for PROBLEM. ERROR=200 -- This is going to have adverse effects. PANIC=300 -- We're dead! - mx.Log: SYSTEM_DEBUG SYSTEM_INFO SYSTEM_UNIMPORTANT SYSTEM_MESSAGE SYSTEM_WARNING SYSTEM_IMPORTANT SYSTEM_CANCEL SYSTEM_ERROR SYSTEM_PANIC SYSTEM_FATAL The current proposal is to copy log4j. XXX I suppose I could see adding zLOG's "TRACE" level, but I am not sure of the usefulness of others. Static Logging Methods (as per Syslog)? Both zLOG and Syslog provide module-level logging functions rather (or in addition to) logging methods on a created Logger object. XXX Is this something that is deemed worth including? Pros: - It would make the simplest case shorter: import logging logging.error("Something is wrong") instead of import logging log = logging.getLogger("") log.error("Something is wrong") Cons: - It provides more than one way to do it. - It encourages logging without a channel name, because this mechanism would likely be implemented by implicitly logging on the root (and nameless) logger of the hierarchy. References [1] java.util.logging http://java.sun.com/j2se/1.4/docs/guide/util/logging/ [2] log4j: a Java logging package http://jakarta.apache.org/log4j/docs/index.html [3] Protomatter's Syslog http://protomatter.sourceforge.net/1.1.6/index.html http://protomatter.sourceforge.net/1.1.6/javadoc/com/protomatter/syslog/sys… [4] MAL mentions his mx.Log logging module: http://mail.python.org/pipermail/python-dev/2002-February/019767.html [5] Jeff Bauer's Mr. Creosote http://starship.python.net/crew/jbauer/creosote/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil fill-column: 70 End: -- Trent Mick TrentM(a)ActiveState.com

6 5

Re: [Python-Dev] For review: PEP 285: Adding a bool type
by Guido van Rossum March 8, 2002

March 8, 2002

> > Absolutely not. I want True == 2 to be False. > > Interesting. so: > operator.truth(2) would be True Yes, this would do exactly the same as bool(2). > but: > 2 == True would be False Just as in C99 (the new updated C standard, not yet in actual use). > and: > bool(2) would be True > > Once again, casting, coercion, comparisons and truth value rear their > fuzzy heads. For the first time, we'll have a standard way to cast an arbitrary value to a normalized truth value. Currently we write "not not x" for that. Now *that* is ugly. Also, you'll be able to use 'is' and 'is not', as follows: if bool(x) is True: ... if bool(y) is not True: ... Not that I endorse this (I still think that anybody who takes a boolean result and then compares it to True is crazy), but it reads like English. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

[Fwd: Re: [Python-Dev] For review: PEP 285: Adding a bool type]
by David Ascher March 8, 2002

March 8, 2002

[resent - forgot python-dev in my reply to guido] Guido van Rossum wrote: > Absolutely not. I want True == 2 to be False. Intersesting. so: operator.truth(2) would be True but: 2 == True would be False and: bool(2) would be True Once again, casting, coercion, comparisons and truth value rear their fuzzy heads. --david

1 0

Bug#137399: patch for urllib2.py: fix behavior with proxies
by Chris Lawrence March 8, 2002

March 8, 2002

Package: python2.1, python2.2 Version: 2.1.2-2 Severity: normal File: /usr/lib/python2.1/urllib2.py Tags: patch upstream The following patch against Python 2.1 fixes some problems with the urllib2 module when used with proxies; in particular, if $http_proxy="http://user:passwd@host:port/" is used. It also generates the correct Host header for proxy requests (some proxies, such as oops, get confused otherwise, despite RFC 2616 section 5.2 which says they are to ignore it in the case of a full URL on the request line). I believe this patch will apply against 2.2 and CVS as well, but I haven't checked yet. --- /usr/lib/python2.1/urllib2.py Fri Jan 18 11:07:32 2002 +++ urllib2.py Fri Mar 8 13:22:14 2002 @@ -478,11 +478,14 @@ def proxy_open(self, req, proxy, type): orig_type = req.get_type() type, r_type = splittype(proxy) - host, XXX = splithost(r_type) - if '@' in host: - user_pass, host = host.split('@', 1) + if '@' in r_type: + user_pass, r_type = r_type.split('@', 1) + if user_pass[:2] == '//': + user_pass = user_pass[2:] + r_type = '//'+r_type user_pass = base64.encodestring(unquote(user_pass)).strip() req.add_header('Proxy-Authorization', 'Basic '+user_pass) + host, XXX = splithost(r_type) host = unquote(host) req.set_proxy(host, type) if orig_type == type: @@ -780,6 +783,7 @@ hexrep.append(hex(n)[-1]) return ''.join(hexrep) +_selectorexp = re.compile(r'[a-z]+://([^/]+)/?', re.I) class AbstractHTTPHandler(BaseHandler): @@ -787,23 +791,29 @@ host = req.get_host() if not host: raise URLError('no host given') + selector = req.get_selector() try: h = http_class(host) # will parse host:port if req.has_data(): data = req.get_data() - h.putrequest('POST', req.get_selector()) + h.putrequest('POST', selector) if not req.headers.has_key('Content-type'): h.putheader('Content-type', 'application/x-www-form-urlencoded') if not req.headers.has_key('Content-length'): h.putheader('Content-length', '%d' % len(data)) else: - h.putrequest('GET', req.get_selector()) + h.putrequest('GET', selector) except socket.error, err: raise URLError(err) - h.putheader('Host', host) + m = _selectorexp.match(selector) + if m: + h.putheader('Host', m.group(1)) + else: + h.putheader('Host', host) + for args in self.parent.addheaders: h.putheader(*args) for k, v in req.headers.items():

2 1