Mailman 3 February 2002 - Python-Dev

Incorporating Expat
by Martin v. Loewis 12 Feb '02

12 Feb '02

At IPC10, I discussed with Fred a strategy for incorporating Expat 1.95.2 into Python. I've now implemented most of this, consisting of: - adding the lib/ directory of Expat as Modules/expat - changing setup.py to build the included Expat library into the extension module - likewise for PCbuild/pyexpat.dsp - dropping support for older Expat versions from pyexpat (support for older Python version is still maintained) AFAICT, the only missing part is to add the relevant changes to Modules/Setup.in, and to update various documentation files. Please make sure to pick up new directories when updating your CVS sandbox. If you find any problems with that change, please let me know. Regards, Martin

1 0

RE: Option Parsing Libraries
by Mark McEahern 12 Feb '02

12 Feb '02

[Paul Prescod] > If you have a competitive library, or suggestions for changes to Optik, > please forward your comments to python-dev mailing list > (python-dev(a)python.org). I love optik. We use it for all of our option parsing. I have one feature request related to error handling. I've attached sample code below that shows a common thing I end up doing: raising an error if a required option is missing. I guess it's not really an option, then, is it? <wink> Anyway, I searched optik's documentation for some way to "inspect" the options collection itself for the original information used when creating the option. In the code below, you'll notice the requiredVar method takes a description parameter. It would be nice to be able to do something like this instead: if var is None: parser.error("Missing: %s" % options.var.description) In fact, it would seem that this itself is so common it would be "built-in" to optik itself. So that all I have to do is declare an option as required when I add it with add_option? Thanks, // mark #! /usr/bin/env python # testError.py from optik import OptionParser def requiredVar(parser, options, var, description): """Raise a parser error if var is None.""" if var is None: # Here's where it'd be nice to have access to the attributes of the # options; at the very least, so I could say which option is missing # without having to pass in the description. parser.error("Missing: %s" % description) def parseCommandLine(): """Parse the command line options and return (options, args).""" usage = """usage: %prog [options] Testing optik's error handling. """ parser = OptionParser(usage) parser.add_option("-f", "--file", type="string", dest="filename", metavar="FILE", help="read data from FILE", default=None) options, args = parser.parse_args() requiredVar(parser, options, options.filename, "filename") def main(): options, args = parseCommandLine() if __name__=="__main__": main()

3 3

Re: RFC: Option Parsing Libraries
by Ben Wolfson 12 Feb '02

12 Feb '02

I have an option-parsing module at http://home.uchicago.edu/~wolfson/Python/arglist.py ; from reading Optik's docs it isn't as featureful but it does answer the list given in http://mail.python.org/pipermail/python-dev/2002-February/019937.html. I haven't used Optik, but my module seems simpler to use. It supports arbitrary callbacks, can be strongly typed (insofar as passing, eg the int function as a callback will generate an error if the value isn't int-able), recognizes the identity of short and long forms, and has a reasonable range of actions--if an option has no argument, by default it records if it appeared and how often; if it does, it can append multiple occurences to a list or keep only the last. Callbacks aren't as flexible as Optik's (they are functions of one argument) but with nested scopes that probably wouldn't be terribly problematic. I'm currently in the process of re-writing arglist.py in my spare time since the code is rather messy, so if it's seen as a contender I could add features. -- BTR BEN WOLFSON HAS RUINED ROCK MUSIC FOR A GENERATION -- Crgre Jvyyneq

1 0

Re: [Python-Dev] patch: speed up name access by up to 80%
by Neal Norwitz 12 Feb '02

12 Feb '02

Oren Tirosh wrote: > > On Mon, Feb 11, 2002 at 05:26:01PM -0500, Neal Norwitz wrote: > > I tried this patch (*) by running the regression tests: > > > > make && time ./python -E -tt Lib/test/regrtest.py > > > > All the expected tests passed and there were no failures, this is good. > > The bad news is that it was slower. It took 42 user seconds longer > > with the patch than without. > > I tried this and the results were identical for both version (+-1 second) > The regression tests most of their time on things like threads, sockets, > signals and stuff like that which is barely affected by this patch and > has a lot of other sources of variance. Some subset of the regression > tests may be good as a benchmark, though. > > I also got very strange results (some faster, some slower) with the > python2.2 RPM package and didn't start to get consistent results until I > used a freshly compiled interpreter for both the reference and DUT. I want > to see how the package was compiled and why it got such strange results. I was surprised by the results. I am using the latest version from CVS, plus I have some outstanding changes that would be unlikely to cause a conflict. They deal with removing consecutive line numbers (there is a patch on SF) and speeding up conditionals. Nothing that is remotely close to dictionaries. Here are all the options from the compile: gcc -c -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fkeep-inline-functions -fno-inline -fprofile-arcs -ftest-coverage -I. -I./Include -DHAVE_CONFIG_H It's possible that no inlining is hurting, but I would still have expected your patch to be faster than without. I also wouldn't expect the test coverage to be hurting (-fprofile-arcs -ftest-coverage), but that is possible. It would be nice if someone else could duplicate either of our results. Neal

1 0

Want to co-design and implement a logging module?
by Guido van Rossum 11 Feb '02

11 Feb '02

I'd like to see a logging module in the standard Python library. Is anybody interested in helping spec out requirements and work on an implementation? Some ideas from Zope's zLOG module should probably go into it (it should eventually be a replacement for that), and some from log4j (http://jakarta.apache.org/log4j/docs/). Any takers? --Guido van Rossum (home page: http://www.python.org/~guido/)

8 8

Proposed standard module: Optik
by Greg Ward 11 Feb '02

11 Feb '02

Hi all -- I would like to propose adding my Optik module to the standard library. Optik is the all-singing, all-dancing, featureful, extensible, well-documented option-parsing module that I have always wanted. Now I have it, and I like it -- so much that I'd like it to just always be there whenever I fire up Python (2.3 or greater, of course). Please take a look at http://optik.sourceforge.net/ for the whole story, including all the documentation and code via CVS. Note that Optik is currently distributed as a package with three modules; it's a hair over 1000 lines of text (563 lines of code), though, so could easily be munged into a single file if that's preferred for the standard library. Two good arguments for adding Optik to the standard library: * David Goodger wants to use it for the standard doc-processing tools * I'd like to use it in the Distutils (and ditch distutils.fancy_getopt) Greg -- Greg Ward - Unix bigot gward(a)python.net http://starship.python.net/~gward/ No man is an island, but some of us are long peninsulas.

5 11

RE: RFC: Option Parsing Libraries
by Gerson.Kurz＠t-online.de 11 Feb '02

11 Feb '02

I will get beat for this, but: can it be optionally non-case-sensitive? I know, I know, in time-honoured unix-tradition a commandline should be dangerous and unforgiving in use, but still, please? Also, I've just scanned the specs and didn't find some "rest-of-the-commandline-whatever-that-is" option. As in: filename options file1 file2 ... filen Other than that, it looks pretty neat.

2 1

PYC Magic
by M.-A. Lemburg 11 Feb '02

11 Feb '02

FYI, I've bumped the PYC magic in a non-standard way (the old standard broke on 2002-01-01); please review: import.c: """ /* New way to come up with the low 16 bits of the magic number: (YEAR-1995) * 10000 + MONTH * 100 + DAY where MONTH and DAY are 1-based. XXX Whatever the "old way" may have been isn't documented. XXX This scheme breaks in 2002, as (2002-1995)*10000 = 70000 doesn't fit in 16 bits. XXX Later, sometimes 1 gets added to MAGIC in order to record that the Unicode -U option is in use. IMO (Tim's), that's a Bad Idea (quite apart from that the -U option doesn't work so isn't used anyway). XXX MAL, 2002-02-07: I had to modify the MAGIC due to a fix of the UTF-8 encoder (it previously produced invalid UTF-8 for unpaired high surrogates), so I simply bumped the month value to 20 (invalid month) and set the day to 1. This should be recognizable by any algorithm relying on the above scheme. Perhaps we should simply start counting in increments of 10 from now on ?! Known values: Python 1.5: 20121 Python 1.5.1: 20121 Python 1.5.2: 20121 Python 2.0: 50823 Python 2.0.1: 50823 Python 2.1: 60202 Python 2.1.1: 60202 Python 2.1.2: 60202 Python 2.2: 60717 Python 2.3a0: 62001 */ #define MAGIC (62001 | ((long)'\r'<<16) | ((long)'\n'<<24)) """ -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

4 10

Re: [Python-Dev] proposal: add basic time type to the standard library
by Stephan Richter 11 Feb '02

11 Feb '02

Hello everyone, what a coincidence. I just was discussing this issue with Jason O. today. Here is my original mail: Hey Jason, I also want to start to think about a DateTime module. PostGres has a nice discussion of their impementation: http://www.ca.postgresql.org/users-lounge/docs/7.1/user/datatype-datetime.h… Here is Java's stuff on it: http://www3.cbu.edu/sciences/jdk1.3/docs/api/java/sql/Date.html Low Level Data Types: Date Time DateTime TimeStamp - Timestamps are always in UTC. * Intervals can be added or subtracted from themselves and the types above. DateInterval TimeInterval DateTimeInterval TimeStampInterval Notes: - The basic data type must be as small as possible, so that applications that save these types often (i.e. ZODB/Persistent) then it should not increase the amount of data by much. - We should then have high-level classes that put in all the functionality, such as lower-level in/output. I think high-level in/output should be handled by functions inside the module, such as getDateTimeFromString(str, someI18NspecificInfo=None). - We need flexible i18n support!!! This is very important, especially for Zope. By default the system should come with a gettext implementation, but I would like to have the module generic enough that we can define other types of translation and localization mechanisms. Mh, the more I think of it, the more I think we will end up building our own stuff and then exposing that via an API. - The parsing of Date, Time and DateTimes as well as their Intervals (PostGreSQL has some very nice ways for that) should be tremendously flexible. I am thinking here about a plugin-type architecture, where you can create your own plugins for parsing. For example, while the "." notation was reserved for the European Date Formats until now, more and more American companies (which are totally ignorant that there might be another country besides the US in the world) use this notation to write the American Date Format this way too. Therefore we need to have a mechanism to switch between the two. I thought of some sort of a list of regex expressions which try to resolve a string. Oh yeah, we need internationalization here as well of course, even though the parser should be generic enough. - The tough part will be time zones. I am almost thinking that we need our own object for handling that. Timezones are horribly complex, but we need to handle them well. I know Zope's current DateTime implementation has a good handle on that, even though I think the code is horrible (sorry Jim). - A professor just mentioned that we should also handle daylight saving. This is not even that trivial, but I agree with him; there needs to be support for that, even though most apps handle that via the time zone, which is ok for the numeric version, but not if you say "CST" for example. PS: Jim, I cc'ed you so that you might be able to comment in some of the points I made. FYI, Jason and I think about implementing a DateTime module for Python in general, which is small and sweet. We are shooting for our calendar system only. Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Student Web2k - Web Design/Development & Technical Project Management

2 6

Re: [Python-Dev] Want to co-design and implement a logging module?
by Michael Hudson 11 Feb '02

11 Feb '02

Greg Ward <gward(a)python.net> writes: > On 11 February 2002, Michael Hudson said: > > But I believe that 1.5.2 compatibility is still relavent for > > distutils > > I'm still catching up on distutils-sig traffic from the past year, so I > don't want to overcommit myself here... but I've been thinking that we > (I) should do one last Distutils release that is 1.5.2 compatible, and > then we can decide if future Distutils releases will stick to > 2.0-compatibility, or are allowed to require the version of Python that > they go with. I;m not sure that idea will get widespread support. > However, please *don't* everyone jump in and start a thread about this > now. I'll take it up on distutils-sig when I've caught up. But I'll wait until you get caught up. > > I had a go at implementing a very KISS approach to distutils logging > > this morning and found what I was doing conflicted horribly with > > distutils' current practice, so I stopped. > > Probably because the Distutils current practice is an ill-thought-out > mishmash. That'll have to be fixed first, I suspect. Sorry. ;-( It was more to do with options processing (the fact that basically speaking all options translate to attributes on some object) than logging. I suspect I could have used Optik more easily... I'm also not sure how politic it would be to take an axe to the interfaces of the various *util modules. Cheers, M. -- You sound surprised. We're talking about a government department here - they have procedures, not intelligence. -- Ben Hutchings, cam.misc

1 0

Python-Dev February 2002