Greg Stein recently reminded me that he was holding off on 1.6 patches
because he was under the impression that I wasn't accepting them yet.
The situation is rather more complicated than that. There are a great
deal of things that need to be done, and for many of them I'd be most
happy to receive patches! For other things, however, I'm still in the
requirements analysis phase, and patches might be premature (e.g., I
want to redesign the import mechanisms, and while I like some of the
prototypes that have been posted, I'm not ready to commit to any
How do you know for which things I'm ready for patches? Ask me. I've
tried to make lists before, and there are probably some hints in the
TODO FAQ wizard as well as in the "requests" section of the Python
Greg also suggested that I might receive more patches if I opened up
the CVS tree for checkins by certain valued contributors. On the one
hand I'm reluctant to do that (I feel I have a pretty good track
record of checking in patches that are mailed to me, assuming I agree
with them) but on the other hand there might be something to say for
this, because it gives contributors more of a sense of belonging to
the inner core. Of course, checkin privileges don't mean you can
check in anything you like -- as in the Apache world, changes must be
discussed and approved by the group, and I would like to have a veto.
However once a change is approved, it's much easier if the contributor
can check the code in without having to go through me all the time.
A drawback may be that some people will make very forceful requests to
be given checkin privileges, only to never use them; just like there
are some members of python-dev who have never contributed. I
definitely want to limit the number of privileged contributors to a
very small number (e.g. 10-15).
One additional detail is the legal side -- contributors will have to
sign some kind of legal document similar to the current (wetsign.html)
release form, but guiding all future contributions. I'll have to
discuss this with CNRI's legal team.
Greg, I understand you have checkin privileges for Apache. What is
the procedure there for handing out those privileges? What is the
procedure for using them? (E.g. if you made a bogus change to part of
Apache you're not supposed to work on, what happens?)
I'm hoping for several kind of responses to this email:
- uncontroversial patches
- questions about whether specific issues are sufficiently settled to
start coding a patch
- discussion threads opening up some issues that haven't been settled
yet (like the current, very productive, thread in i18n)
- posts summarizing issues that were settled long ago in the past,
requesting reverification that the issue is still settled
- suggestions for new issues that maybe ought to be settled in 1.6
- requests for checkin privileges, preferably with a specific issue or
area of expertise for which the requestor will take responsibility
--Guido van Rossum (home page: http://www.python.org/~guido/)
[ taking the liberty to CC: this back to python-dev ]
On Fri, 19 Nov 1999, David Ascher wrote:
> > > (2) a file in a directory that's on sys.path can be a zip/jar file;
> > > its contents will be considered as a package (note that this is
> > > different from (1)!)
> > No problem. This will slow things down, as a stat() for *.zip and/or *.jar
> > must be done, in addition to *.py, *.pyc, and *.pyo.
> Aside: it strikes me that for Python programs which import lots of files,
> 'front-loading' the stat calls could make sense. When you first look at a
> directory in sys.path, you read the entire directory in memory, and
> successive imports do a stat on the directory to see if it's changed, and
> if not use the in-memory data. Or am I completely off my rocker here?
Not at all. I thought of this last night after my email. Since the
Importer can easily retain state, it can hold a cache of the directory
listings. If it doesn't find the file in its cached state, then it can
reload the information from disk. If it finds it in the cache, but not on
disk, then it can remove the item from its cache.
The problem occurs when you path is [A, B], the file is in B, and you add
something to A on-the-fly. The cache might direct the importer at B,
missing your file.
Of course, with the appropriate caveats/warnings, the system would work
quite well. It really only breaks during development (which is one reason
why I didn't accept some caching changes to imputil from MAL; but that
was for the Importer in there; Python's new Importer could have a cache).
I'm also not quite sure what the cost of reading a directory is, compared
to issuing a bunch of stat() calls. Each directory read is an
opendir/readdir(s)/closedir. Note that the DBM approach is kind of
similar, but will amortize this cost over many processes.
Greg Stein, http://www.lyra.org/
I've updated imputil... The main changes is that I added SysPathImporter
and BuiltinImporter. I also did some restructing to help with
bootstrapping the module (remove dependence on os.py).
For testing a revamped Python import system, you can importing the thing
and call imputil._test_revamp() to set it up. This will load normal,
builtin, and frozen modules via imputil. Dynamic modules are still
handled by Python, however.
I ran a timing comparisons of importing all modules in /usr/lib/python1.5
(using standard and imputil-based importing). The standard mechanism can
do it in about 8.8 seconds. Through imputil, it does it in about 13.0
seconds. Note that I haven't profiled/optimized any of the Importer stuff
The point about dynamic modules actually discovered a basic problem that I
need to resolve now. The current imputil assumes that if a particular
Importer loaded the top-level module in a package, then that Importer is
responsible for loading all other modules within that package. In my
particular test, I tried to import "xml.parsers.pyexpat". The two package
modules were handled by SysPathImporter. The pyexpat module is a dynamic
load module, so it is *not* handled by the Importer -- bam. Failure.
Basically, each part of "xml.parsers.pyexpat" may need to use a different
Off to ponder,
Greg Stein, http://www.lyra.org/
My first Python-Dev post. :-)
>We had some discussion a while back about enabling thread support by
>default, if the underlying OS supports it obviously.
What's the consensus about Python microthreads -- a likely candidate
for incorporation in 1.6 (or later)?
Also, we have a couple minor convenience functions for Python in an
MSDEV environment, an exposure of OutputDebugString for writing to
the DevStudio log window and a means of tripping DevStudio C/C++ layer
breakpoints from Python code (currently experimental). The msvcrt
module seems like a likely candidate for these, would these be
Origin Systems, Inc.
Hello Python Developers!
Thursday January 27 2000, the final day of the 8th International
Python Conference is Developers' Day, where Python hackers get
together to discuss and reach agreements on the outstanding issues
facing Python. This is also your once-a-year chance for face-to-face
interactions with Python's creator Guido van Rossum and other
experienced Python developers.
To make Developers' Day a success, we need you! We're looking for a
few good champions to lead topic sessions. As a champion, you will
choose a topic that fires you up and write a short position paper for
publication on the web prior to the conference. You'll also prepare
introductory material for the topic overview session, and lead a 90
minute topic breakout group.
We've had great champions and topics in previous years, and many
features of today's Python had their start at past Developers' Days.
This is your chance to help shape the future of Python for 1.6,
2.0 and beyond.
If you are interested in becoming a topic champion, you must email me
by Wednesday December 15, 1999. For more information, please visit
the IPC8 Developers' Day web page at
This page has more detail on schedule, suggested topics, important
dates, etc. To volunteer as a champion, or to ask other questions,
you can email me at bwarsaw(a)python.org.
Before hooking on to some more PathBuiltinImporters ;-), I'd like
to spawn a thread leading in a different direction...
There has been some discussion on what we really expect of the
import mechanism to be able to do. Here's a summary of what I
think we need:
* compatibility with the existing import mechanism
* imports from library archives (e.g. .pyl or .par-files)
* a modified intra package import lookup scheme (the thingy
which I call "walk-me-up-Scotty" patch -- see previous posts)
And for some fancy stuff:
* imports from URLs (e.g. these could be put on the path for
automatic inclusion in the import scan or be passed explicitly
* a (file based) static lookup cache to enhance lookup
performance which is enabled via a command line switch
(rather than being enabled per default), so that the
user can decide whether to apply this optimization or
The point I want to make is: there aren't all that many features
we are really looking for, so why not incorporate these into
the builtin importer and only *then* start thinking about
schemes for hooks, managers, etc. ?!
Y2000: 37 days left
Python Pages: http://www.lemburg.com/python/
--- Mark Hammond <mhammond(a)skippinet.com.au> wrote:
> > Pretty quiet around here lately...
> My guess is that most positions and opinions have
> been covered. It is
> now probably time for less talk, and more code!
> It is time to start an implementation plan? Do we
> start with /F's
> Unicode implementation (which /G *smirk* seemed to
> approve of)? Who
> does what? When can we start to play with it?
> And a key point that seems to have been thrust in
> our faces at the
> start and hardly mentioned recently - does the
> proposal as it stands
> meet our sponsor's (HP) requirements?
I had a long chat with them on Friday :-) They want
it done, but nobody is actively working on it now as
far as I can tell, and they are very busy.
The per-thread thing was a red herring - they just
want to be able to do (for example) web servers
handling different encodings from a central unicode
database, so per-output-stream works just fine.
They will be at IPC8; I'd suggest that a round of
prototyping, we insist they read it and then discuss
it at IPC8, and be prepared to rework things
thereafter are important. Hopefully then we'll have a
plan on how to tackle the much larger (but less
interesting to python-dev) job of writing and
verifying all the codecs and utilities.
Robinson Analytics Ltd.
My opinions are the official policy of Robinson Analytics Ltd.
They just vary from day to day.
Do You Yahoo!?
Thousands of Stores. Millions of Products. All in one place.
Yahoo! Shopping: http://shopping.yahoo.com
I've uploaded a new version of the proposal which incorporates
a lot of what has been discussed on the list.
Thanks to everybody who helped so far. Note that I have extended
the list of references for those who want to join in, but are
in need of more background information.
The latest version of the proposal is available at:
Older versions are available as:
Some POD (points of discussion) that are still open:
· support for line breaks (see
· support for case conversion:
Problems: string lengths can change due to multiple
characters being mapped to a single new one, capital letters
starting a word can be different than ones occurring in the
middle, there are locale dependent deviations from the standard
· support for numbers, digits, whitespace, etc.
· support (or no support) for private code point areas
· should Unicode objects support %-formatting ?
One possibility would be to emulate this via strings and
s = '%s %i abcäöü' # a Latin-1 encoded string
t = (u,3)
# Convert Latin-1 s to a <default encoding> string
s1 = unicode(s,'latin-1').encode()
# The '%s' will now add u in <default encoding>
s2 = s1 % t
# Finally, convert the <default encoding> encoded string to Unicode
u1 = unicode(s2)
· specifying file wrappers:
Open issues: what to do with Python strings
fed to the .write() method (may need to know the encoding of the
strings) and when/if to return Python strings through the .read()
Perhaps we need more than one type of wrapper here.
Y2000: 49 days left
Python Pages: http://www.lemburg.com/python/
I can't find the message, but it seems that at some point you mentioned
wanting to break out importdl.c into separate files. The configure process
could then select the appropriate one to use for the platform.
Sounded great until I looked at importdl.c. There are a 13 variants of
dynamic loading. That would imply 13 separate files/modules.
I'd be happy to break these out, but are you actually interested in that
many resulting modules? If so, then any suggestions for naming?
(e.g. aix_dynload, win32_dynload, mac_dynload)
Here are the variants:
* NeXT, using FVM shlibs (USE_RLD)
* NeXT, using frameworks (USE_DYLD)
* dl / GNU dld (USE_DL)
* SunOS, IRIX 5 shared libs (USE_SHLIB)
* AIX dynamic linking (_AIX)
* Win32 platform (MS_WIN32)
* Win16 platform (MS_WIN16)
* OS/2 dynamic linking (PYOS_OS2)
* Mac CFM (USE_MAC_DYNAMIC_LOADING)
* HP/UX dyn linking (hpux)
* NetBSD shared libs (__NetBSD__)
* FreeBSD shared libs (__FreeBSD__)
* BeOS shared libs (__BEOS__)
Could I suggest a new top-level directory in the Python distribution named
"Platform"? Move BeOS, PC, and PCbuild in there (bring back Mac?). Add new
directories for each of the above platforms and move the appropriate
portion of importdl.c into there as a Python C Extension Module. (the
module would still be statically linked into the interpreter!)
./configure could select the module and write a Setup.dynload, much like
it does with Setup.thread.
Greg Stein, http://www.lyra.org/
Just got a note about a paper on a new way of dealing with threads, as
presented to the Perl-Porters list. The idea is described in:
I have no time to dive in, comment, or even judge the relevance of this,
but perhaps someone else on this list wishes to check it out.
The author of this is Greg London <bartels(a)pixelmagic.com>.