Mailman 3 November 1999 - Python-Dev

Python 1.6 status
by Guido van Rossum 07 Dec '99

07 Dec '99

Greg Stein recently reminded me that he was holding off on 1.6 patches because he was under the impression that I wasn't accepting them yet. The situation is rather more complicated than that. There are a great deal of things that need to be done, and for many of them I'd be most happy to receive patches! For other things, however, I'm still in the requirements analysis phase, and patches might be premature (e.g., I want to redesign the import mechanisms, and while I like some of the prototypes that have been posted, I'm not ready to commit to any specific implementation). How do you know for which things I'm ready for patches? Ask me. I've tried to make lists before, and there are probably some hints in the TODO FAQ wizard as well as in the "requests" section of the Python Bugs List. Greg also suggested that I might receive more patches if I opened up the CVS tree for checkins by certain valued contributors. On the one hand I'm reluctant to do that (I feel I have a pretty good track record of checking in patches that are mailed to me, assuming I agree with them) but on the other hand there might be something to say for this, because it gives contributors more of a sense of belonging to the inner core. Of course, checkin privileges don't mean you can check in anything you like -- as in the Apache world, changes must be discussed and approved by the group, and I would like to have a veto. However once a change is approved, it's much easier if the contributor can check the code in without having to go through me all the time. A drawback may be that some people will make very forceful requests to be given checkin privileges, only to never use them; just like there are some members of python-dev who have never contributed. I definitely want to limit the number of privileged contributors to a very small number (e.g. 10-15). One additional detail is the legal side -- contributors will have to sign some kind of legal document similar to the current (wetsign.html) release form, but guiding all future contributions. I'll have to discuss this with CNRI's legal team. Greg, I understand you have checkin privileges for Apache. What is the procedure there for handing out those privileges? What is the procedure for using them? (E.g. if you made a bogus change to part of Apache you're not supposed to work on, what happens?) I'm hoping for several kind of responses to this email: - uncontroversial patches - questions about whether specific issues are sufficiently settled to start coding a patch - discussion threads opening up some issues that haven't been settled yet (like the current, very productive, thread in i18n) - posts summarizing issues that were settled long ago in the past, requesting reverification that the issue is still settled - suggestions for new issues that maybe ought to be settled in 1.6 - requests for checkin privileges, preferably with a specific issue or area of expertise for which the requestor will take responsibility --Guido van Rossum (home page: http://www.python.org/~guido/)

14 54

Re: [Python-Dev] Import redesign [LONG]
by Greg Stein 02 Dec '99

02 Dec '99

[ taking the liberty to CC: this back to python-dev ] On Fri, 19 Nov 1999, David Ascher wrote: > > > (2) a file in a directory that's on sys.path can be a zip/jar file; > > > its contents will be considered as a package (note that this is > > > different from (1)!) > > > > No problem. This will slow things down, as a stat() for *.zip and/or *.jar > > must be done, in addition to *.py, *.pyc, and *.pyo. > > Aside: it strikes me that for Python programs which import lots of files, > 'front-loading' the stat calls could make sense. When you first look at a > directory in sys.path, you read the entire directory in memory, and > successive imports do a stat on the directory to see if it's changed, and > if not use the in-memory data. Or am I completely off my rocker here? Not at all. I thought of this last night after my email. Since the Importer can easily retain state, it can hold a cache of the directory listings. If it doesn't find the file in its cached state, then it can reload the information from disk. If it finds it in the cache, but not on disk, then it can remove the item from its cache. The problem occurs when you path is [A, B], the file is in B, and you add something to A on-the-fly. The cache might direct the importer at B, missing your file. Of course, with the appropriate caveats/warnings, the system would work quite well. It really only breaks during development (which is one reason why I didn't accept some caching changes to imputil from MAL; but that was for the Importer in there; Python's new Importer could have a cache). I'm also not quite sure what the cost of reading a directory is, compared to issuing a bunch of stat() calls. Each directory read is an opendir/readdir(s)/closedir. Note that the DBM approach is kind of similar, but will amortize this cost over many processes. Cheers, -g -- Greg Stein, http://www.lyra.org/

5 9

updated imputil
by Greg Stein 02 Dec '99

02 Dec '99

I've updated imputil... The main changes is that I added SysPathImporter and BuiltinImporter. I also did some restructing to help with bootstrapping the module (remove dependence on os.py). For testing a revamped Python import system, you can importing the thing and call imputil._test_revamp() to set it up. This will load normal, builtin, and frozen modules via imputil. Dynamic modules are still handled by Python, however. I ran a timing comparisons of importing all modules in /usr/lib/python1.5 (using standard and imputil-based importing). The standard mechanism can do it in about 8.8 seconds. Through imputil, it does it in about 13.0 seconds. Note that I haven't profiled/optimized any of the Importer stuff (yet). The point about dynamic modules actually discovered a basic problem that I need to resolve now. The current imputil assumes that if a particular Importer loaded the top-level module in a package, then that Importer is responsible for loading all other modules within that package. In my particular test, I tried to import "xml.parsers.pyexpat". The two package modules were handled by SysPathImporter. The pyexpat module is a dynamic load module, so it is *not* handled by the Importer -- bam. Failure. Basically, each part of "xml.parsers.pyexpat" may need to use a different Importer... Off to ponder, -g -- Greg Stein, http://www.lyra.org/

8 23

RE: [Python-Dev] Another 1.6 wish
by Asbahr, Jason 01 Dec '99

01 Dec '99

My first Python-Dev post. :-) >We had some discussion a while back about enabling thread support by >default, if the underlying OS supports it obviously. What's the consensus about Python microthreads -- a likely candidate for incorporation in 1.6 (or later)? Also, we have a couple minor convenience functions for Python in an MSDEV environment, an exposure of OutputDebugString for writing to the DevStudio log window and a means of tripping DevStudio C/C++ layer breakpoints from Python code (currently experimental). The msvcrt module seems like a likely candidate for these, would these be welcome additions? Thanks, Jason Asbahr Origin Systems, Inc. jasbahr(a)origin.ea.com

5 7

CFP Developers' Day - 8th International Python Conference
by bwarsaw＠python.org 30 Nov '99

30 Nov '99

Hello Python Developers! Thursday January 27 2000, the final day of the 8th International Python Conference is Developers' Day, where Python hackers get together to discuss and reach agreements on the outstanding issues facing Python. This is also your once-a-year chance for face-to-face interactions with Python's creator Guido van Rossum and other experienced Python developers. To make Developers' Day a success, we need you! We're looking for a few good champions to lead topic sessions. As a champion, you will choose a topic that fires you up and write a short position paper for publication on the web prior to the conference. You'll also prepare introductory material for the topic overview session, and lead a 90 minute topic breakout group. We've had great champions and topics in previous years, and many features of today's Python had their start at past Developers' Days. This is your chance to help shape the future of Python for 1.6, 2.0 and beyond. If you are interested in becoming a topic champion, you must email me by Wednesday December 15, 1999. For more information, please visit the IPC8 Developers' Day web page at <http://www.python.org/workshops/2000-01/devday.html> This page has more detail on schedule, suggested topics, important dates, etc. To volunteer as a champion, or to ask other questions, you can email me at bwarsaw(a)python.org. -Barry

1 0

Import Design
by M.-A. Lemburg 24 Nov '99

24 Nov '99

Before hooking on to some more PathBuiltinImporters ;-), I'd like to spawn a thread leading in a different direction... There has been some discussion on what we really expect of the import mechanism to be able to do. Here's a summary of what I think we need: * compatibility with the existing import mechanism * imports from library archives (e.g. .pyl or .par-files) * a modified intra package import lookup scheme (the thingy which I call "walk-me-up-Scotty" patch -- see previous posts) And for some fancy stuff: * imports from URLs (e.g. these could be put on the path for automatic inclusion in the import scan or be passed explicitly to __import__) * a (file based) static lookup cache to enhance lookup performance which is enabled via a command line switch (rather than being enabled per default), so that the user can decide whether to apply this optimization or not The point I want to make is: there aren't all that many features we are really looking for, so why not incorporate these into the builtin importer and only *then* start thinking about schemes for hooks, managers, etc. ?! -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 37 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

2 1

RE: [Python-Dev] Unicode Proposal: Version 0.8
by Andy Robinson 24 Nov '99

24 Nov '99

--- Mark Hammond <mhammond(a)skippinet.com.au> wrote: > > Pretty quiet around here lately... > > My guess is that most positions and opinions have > been covered. It is > now probably time for less talk, and more code! > > It is time to start an implementation plan? Do we > start with /F's > Unicode implementation (which /G *smirk* seemed to > approve of)? Who > does what? When can we start to play with it? > > And a key point that seems to have been thrust in > our faces at the > start and hardly mentioned recently - does the > proposal as it stands > meet our sponsor's (HP) requirements? > > Mark. I had a long chat with them on Friday :-) They want it done, but nobody is actively working on it now as far as I can tell, and they are very busy. The per-thread thing was a red herring - they just want to be able to do (for example) web servers handling different encodings from a central unicode database, so per-output-stream works just fine. They will be at IPC8; I'd suggest that a round of prototyping, we insist they read it and then discuss it at IPC8, and be prepared to rework things thereafter are important. Hopefully then we'll have a plan on how to tackle the much larger (but less interesting to python-dev) job of writing and verifying all the codecs and utilities. Andy Robinson ===== Andy Robinson Robinson Analytics Ltd. ------------------ My opinions are the official policy of Robinson Analytics Ltd. They just vary from day to day. __________________________________________________ Do You Yahoo!? Thousands of Stores. Millions of Products. All in one place. Yahoo! Shopping: http://shopping.yahoo.com

1 0

Unicode Proposal: Version 0.4
by M.-A. Lemburg 24 Nov '99

24 Nov '99

I've uploaded a new version of the proposal which incorporates a lot of what has been discussed on the list. Thanks to everybody who helped so far. Note that I have extended the list of references for those who want to join in, but are in need of more background information. The latest version of the proposal is available at: http://starship.skyport.net/~lemburg/unicode-proposal.txt Older versions are available as: http://starship.skyport.net/~lemburg/unicode-proposal-X.X.txt Some POD (points of discussion) that are still open: · support for line breaks (see http://www.unicode.org/unicode/reports/tr13/ ) · support for case conversion: Problems: string lengths can change due to multiple characters being mapped to a single new one, capital letters starting a word can be different than ones occurring in the middle, there are locale dependent deviations from the standard mappings. · support for numbers, digits, whitespace, etc. · support (or no support) for private code point areas · should Unicode objects support %-formatting ? One possibility would be to emulate this via strings and <default encoding>: s = '%s %i abcäöü' # a Latin-1 encoded string t = (u,3) # Convert Latin-1 s to a <default encoding> string s1 = unicode(s,'latin-1').encode() # The '%s' will now add u in <default encoding> s2 = s1 % t # Finally, convert the <default encoding> encoded string to Unicode u1 = unicode(s2) · specifying file wrappers: Open issues: what to do with Python strings fed to the .write() method (may need to know the encoding of the strings) and when/if to return Python strings through the .read() method. Perhaps we need more than one type of wrapper here. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 49 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

3 8

breaking out code for dynamic loading
by Greg Stein 23 Nov '99

23 Nov '99

Guido, I can't find the message, but it seems that at some point you mentioned wanting to break out importdl.c into separate files. The configure process could then select the appropriate one to use for the platform. Sounded great until I looked at importdl.c. There are a 13 variants of dynamic loading. That would imply 13 separate files/modules. I'd be happy to break these out, but are you actually interested in that many resulting modules? If so, then any suggestions for naming? (e.g. aix_dynload, win32_dynload, mac_dynload) Here are the variants: * NeXT, using FVM shlibs (USE_RLD) * NeXT, using frameworks (USE_DYLD) * dl / GNU dld (USE_DL) * SunOS, IRIX 5 shared libs (USE_SHLIB) * AIX dynamic linking (_AIX) * Win32 platform (MS_WIN32) * Win16 platform (MS_WIN16) * OS/2 dynamic linking (PYOS_OS2) * Mac CFM (USE_MAC_DYNAMIC_LOADING) * HP/UX dyn linking (hpux) * NetBSD shared libs (__NetBSD__) * FreeBSD shared libs (__FreeBSD__) * BeOS shared libs (__BEOS__) Could I suggest a new top-level directory in the Python distribution named "Platform"? Move BeOS, PC, and PCbuild in there (bring back Mac?). Add new directories for each of the above platforms and move the appropriate portion of importdl.c into there as a Python C Extension Module. (the module would still be statically linked into the interpreter!) ./configure could select the module and write a Setup.dynload, much like it does with Setup.thread. Cheers, -g -- Greg Stein, http://www.lyra.org/

1 0

New thread ideas in Perl-land
by Jean-Claude Wippler 23 Nov '99

23 Nov '99

Just got a note about a paper on a new way of dealing with threads, as presented to the Perl-Porters list. The idea is described in: http://www.cpan.org/modules/by-authors/id/G/GB/GBARTELS/thread_0001.txt I have no time to dive in, comment, or even judge the relevance of this, but perhaps someone else on this list wishes to check it out. The author of this is Greg London <bartels(a)pixelmagic.com>. -- Jean-Claude

1 0