"select", "signal" aren't UNIX specific. Huh? How not? Can you name a non-UNIX that is providing them? (BeOS wouldn't count, select is broken, and nobody uses signals.) and if you can, is it providing them for something other than "UNIX/POSIX compatibility" "getopt" is used for generic argument processing, so it isn't really UNIX specific.

It's a POSIX.2 function. I consider that UNIX.

And I don't like the name "system" either. But I have no constructive proposals about thos either.

so-i'll-just-shut-up-now-ly y'rs, Z. -- just-picking-nits-ly y'rs, Dan

On Sun, 26 Mar 2000, Moshe Zadka wrote:

...

For example, why text.binhex but text.mail.mime.base64?

Actually, I thought about this (this isn't random at all): base64 encoding is part of the mime standard, together with quoted-printable. Binhex isn't. I don't know if you find it reason enough, and it may be smarter just having a text.encode.{quopri,uu,base64,binhex}

I think i'd like that better, yes.

...

and it's not clear why these all belong under "parse".

These are all used for parsing data (which does not have some pre-written parser). I had problems with the name too...

And parsing is what the "text" package is about anyway. I say move them up. (See the layout in my other message. Notice most of the regular-expression stuff is deprecated anyway, so it's not like there are really that many.)

...

Why doesn't "socket" go under "net"?

What about UNIX domain sockets? Again, no *strong* opinion, though.

Bleck, you're right. Well, i think we just have to pick one or the other here, and i think most people would guess "net" first. (You can think of it as IPC, and file IPC-related things under then "net" category...?)

...

Why does "terminal" belong under "file"?

Because it is (a special kind of file)

Only in Unix. It's Unix that likes to think of all things, including terminals, as files.

I do like the idea of "data-types" package, but it needs to be ironed out a bit.

See my other message for a possible suggested hierarchy...

...

...

internal [...] You mistook my motivation -- I just want unadvertised modules (AKA internal use modules) to live in a carefully segregate section of the namespace. How would this confuse people? No one imports _tkinter or pcre, so no one would notice the change.

I think it makes more sense to classify modules by their topic rather than their exposure. (For example, you wouldn't move deprecated modules to a "deprecated" package.) Keep in mind that (well, at least to me) the main point of any naming hierarchy is to avoid name collisions. "internal" doesn't really help that purpose. You also want to be sure (or as sure as you can) that modules will be obvious to find in the hierarchy. An "internal" package creates a distinction orthogonal to the topic-matter distinction we're using for the rest of the packages, which *potentially* introduces the question "well... is this module internal or not?" for every other module. Yes, admittedly this is only "potentially", but i hope you see the abstract point i'm trying to make...

...

...

locale

I think "locale" belongs under "math" with "fpformat" and the others. It's for numeric formatting.

Only? And anyway, I doubt many people will think like that.

Yeah, it is pretty much only for numeric formatting. The more generic locale stuff seems to be in _locale.

...

They all seem to be Unix-related. How about putting these in a "unix" or "system" package?

"select", "signal" aren't UNIX specific.

Yes, but when they're available on other systems they're an attempt to emulate Unix or Posix functionality, aren't they?

Well, the argument style it processes is not unheard of in other OSes, and it's nice to have command line apps that have a common ui. That's it! "getopt" belongs in the ui package!

I like ui.getopt. It's a pretty good idea. -- ?!ng "I'm not trying not to answer the question; i'm just not answering it." -- Lenore Snell

Responding to an early item in this thread and trying to adapt to later items... Ping wrote: I'm not convinced "mime" needs a separate branch here. (This is the deepest part of the tree, and at three levels small alarm bells went off in my head.) It's not clear that mime should be beneath text/mail. Moshe moved it up a level, but not the way I would have done it. I think the mime stuff still belongs in a separate mime package. I wouldn't just sprinkle the modules under text. I see two possibilities: text>mime net>mime I prefer net>mime, because MIME and its artifacts are used heavily in networked applications where the content being transferred isn't text. -- Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/

On Sat, 25 Mar 2000, Ka-Ping Yee wrote:

Hey, while we're at it... as long as we're renaming modules, what do you all think of getting rid of that "lib" suffix?

+0 -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html http://www.linux.org.il -- we put the penguin in .com

On 25 March 2000, Ka-Ping Yee said:

Okay, here's another shot at it. Notice a few things:

Damn, I started writing a response to Moshe's original proposal -- and *then* saw this massive thread. Oh well. Turns out I still have a few useful things to say: First, any organization scheme for the standard library (or anything else, for that matter) should have a few simple guidelines. Here are two: * "deep hierarchies considered harmful": ie. avoid sub-packages if at all possible * "everything should have a purpose": every top-level package should be describable with a single, clear sentence of plain language. Eg.: net - Internet protocols, data formats, and client/server infrastructure unix - Unix-specific system calls, protocols, and conventions And two somewhat open issues: * "as long as we're renaming...": maybe this would be a good time to standardize naming conventions, eg. "cgi" -> "cgilib" *or* "{http,ftp,url,...}lib" -> "{http,ftp,url}...", "MimeWriter" -> "mimewriter", etc. * "shared namespaces vs system namespaces": the Perl model of "nothing belongs to The System; anyone can add a module in Text:: or Net:: or whatever" works there because Perl doesn't have __init__ files or anything to distinguish module namespaces; they just are. Python's import mechanism would have to change to support this, and the fact that __init__ files may contain arbitrary code makes this feel like a very tricky change to make. Now specific comments...

net urlparse urllib ftplib gopherlib imaplib poplib nntplib smtplib telnetlib httplib cgi

Rename? Either cgi -> cgilib or foolib -> foo?

server BaseHTTPServer CGIHTTPServer SimpleHTTPServer SocketServer asynchat asyncore

This is one good place for a sub-package. It's a also a good place to rename: the convention for Python module names seems to be all-lowercase; and "Server" is redundant when you're in the net.server package. How about: net.server.base_http net.server.cgi_http net.server.simple_http net.server.socket Underscores negotiable. They don't seem to be popular in module names, although sometimes they would be real life-savers.

text

I think "text" should mean "plain old unstructured, un-marked-up ASCII text", where "unstructured, un-marked-up" really means "not structured or marked up in a well-known standard way". Or maybe not. I'm just trying to come up with an excuse for moving xml to top-level, which I think is where it belongs. Maybe the excuse should just be, "XML is really important and visible, and anyways Paul Prescod will raise a stink if it isn't put at top-level in Python package-space".

re # general-purpose parsing

Top-level: this is a fundamental module that should be treated on a par with 'string'. (Well, except for building RE methods into strings... hmmMMmm...maybe... [no, I'm kidding!])

sgmllib htmllib htmlentitydefs

Not sure what to do about these. Someone referred somewhere to a "web" top-level package, which seems to have disappeared. If it reappars, it would be a good place for the HTML modules (not to mention a big chunk of "net") -- this would mainly be for "important and visible" (ie. PR) reasons, rather than sound technical reasons.

xml whatever the xml-sig puts here

Should be top-level.

mail rfc822 mailbox mhlib

"mail" should either be top-level or under "net". (Yes, I *know* it's not a wire-level protocol: that's what net.smtplib is for. But last time I checked, email is pretty useless without a network. And vice-versa.) Or maybe these all belong in a top-level "data" package: I'm starting to warm to that.

bin gzip zlib chunk struct image imghdr colorsys # a bit unsure, but doesn't go anywhere else imageop imgfile rgbimg yuvconvert sound aifc sndhdr toaiff audiodev sunau sunaudio wave audioop sunaudiodev

I agree with Jack: image and sound (audio?) should be top-level. I don't think I like the idea of an intervening "mm" or "multimedia" or "media" or what-have-you package, though. The other stuff in "bin" is kind of a grab-bag: "chunk" and "struct" might belong in the mythical "data" package.

db anydbm whichdb bsddb dbm dbhash dumbdbm gdbm

Yup.

math math # library functions cmath fpectl # type-related fpetest array mpz fpformat # formatting locale bisect # algorithm: also unsure, but doesn't go anywhere else random # randomness whrandom crypt # cryptography md5 rotor sha

Hmmm. "locale" has already been dealt with; obviously it should be top-evel. I think "array" should be top-level or under the mythical "data". Six crypto-related modules seems like enough to justify a top-level "crypt" package, though.

time calendar time tzparse sched timing

Yup.

interp new linecache # handling .py files [...] tabnanny pstats rlcompleter # this might go in "ui"...

I like "python" for this one. (But I'm not sure if tabnanny and rlcompleter belong there.)

security Bastion rexec ihooks

What does ihooks have to do with security?

file dircache path -- a virtual module which would do a from <something>path import * nturl2path macurl2path filecmp fileinput StringIO

Lowercase for consistency?

glob fnmatch stat statcache statvfs tempfile shutil pipes popen2 commands dl

No problem until these last two -- 'commands' is a Unix-specific thing that has very little to do with the filesystem per se, and 'dl' is (as I understand it) deep ju-ju with sharp edges that should probably be hidden away in the 'python' ('sys'?) package. Oh yeah, "dl" should be elsewhere -- "python" maybe? Top-level? Perhaps we need a "deepmagic" package for "dl" and "new"? ;-)

data pickle shelve xdrlib copy copy_reg UserDict UserList pprint repr (cPickle)

Oh hey, it's *not* a mythical package! Guess I didn't read far enough ahead. I like it, but would add more stuff to it (obviously): 'struct', 'chunk', 'array' for starters. Should cPickle be renamed to fastpickle?

threads thread threading Queue

Lowercase?

ui _tkinter curses Tkinter cmd getpass getopt readline

users pwd grp nis

These belong in "unix". Possibly "nis" belongs in "net" -- do any non-Unix OSes use NIS?

sgi al cd cl fl fm gl misc (what used to be sgimodule.c) sv

Should this be "sgi" or "irix"? Ditto for "sun" vs "solaris" if there are a significant number of Sun/Solaris modules. Note that the respective trademark holders might get very antsy about who gets to put names in those namespaces -- that's exactly what happened with Sun, Solaris 8, and Perl. I believe the compromise they arrived at was that the "Solaris::" namespace remains open, but Sun gets the "Sun::" namespace. There should probably be a win32 package, for core registry access stuff if nothing else. There might someday be a "linux" package; it's highly unlikely there would be a "pc" or "alpha" package though. All of those argue over "irix" and "solaris" instead of "sgi" and "sun". Greg

On Sun, 26 Mar 2000, Moshe Zadka wrote:

On Sun, 26 Mar 2000, Greg Stein wrote: ...

...

2) you're pushing too hard. modules do not *have* to go into a package. there are some placements that you've made which are very questionable... it appears they are done for movement's sake rather than for being "right"

Well, I'm certainly sorry I gave that impression -- the reason I wans't "right" wasn't that, it was more my desire to be "fast" -- I wanted to have some proposal out the door, since it is harder to argue about something concrete. The biggest prrof of concept that we all agree is that no one seriously took objections to anything -- there were just some minor nits to pick.

Not something to apologize for! :-) Well, the indicator was the line in your original post about "unhandled modules" and the conversation between you and Ping with statements along the lines of "wasn't sure where to put this." I say just leave it then :-) If a module does not make *obvious* sense to be in a package, then it should not be there. For example: locale. That is not about numbers or about text. It has general utility. If there was an i18n package, then it would go there. Otherwise, don't force it somewhere else. Other packages are similar, so don't single out my comment about locale. Cheers, -g -- Greg Stein, http://www.lyra.org/

On Sun, 26 Mar 2000, Greg Stein wrote:

If a module does not make *obvious* sense to be in a package, then it should not be there. For example: locale. That is not about numbers or about text. It has general utility. If there was an i18n package, then it would go there. Otherwise, don't force it somewhere else. Other packages are similar, so don't single out my comment about locale.

I goofed. I apologize. Moshe and Greg are right: locale isn't just about numbers. I just read the comment at the top of locale.py: "Support for number formatting using the current locale settings" and didn't notice the from _locale import * a couple of lines down. "import locale; dir(locale)" didn't work for me because for some reason there's no _locale built-in on my system (Red Hat 6.1, python-1.5.1-10). So i looked for 'def's and they all looked like they had to do with numeric formatting. My mistake. "locale", at least, belongs at the top level. Other candidates for top-level: bisect # algorithm struct # more general than "bin" or "data" colorsys # not really just for image file formats yuvconvert # not really just for image file formats rlcompleter # not really part of the interpreter dl # not really just about files Alternatively, we could have: ui.rlcompleter, unix.dl (It would be nice, by the way, to replace "bisect" with an "algorithm" module containing some nice pedagogical implementations of things like bisect, quicksort, heapsort, Dijkstra's algorithm etc.) The following also could be left at the top-level, since they seem like applications (i.e. they probably won't get imported by code, only interactively). No strong opinion on this. bdb pdb pyclbr tabnanny profile pstats Also... i was avoiding calling the "unix" package "posix" because we already have a "posix" module. But wait... the proposed tree already contains "math" and "time" packages. If there is no conflict (is there a conflict?) then the "unix" package should probably be named "posix". -- ?!ng "In the sciences, we are now uniquely privileged to sit side by side with the giants on whose shoulders we stand." -- Gerald Holton

Hi! Moshe Zadka wrote:

Yes. That was a hard decision I made, and I'm sort of waiting for Guido to veto it: it would negate the easy backwards compatible path of providing a toplevel module for each module which is moved somewhere else which does "from import *".

If the result of this renaming initiative will be that I can't use import sys, os, time, re, struct, cPickle, parser import Tkinter; Tk=Tkinter; del Tkinter anymore in Python 1.x and instead I have to change this into (for example): form posix import time from text import re from bin import struct from Python import parser from ui import Tkinter; ... ... I would really really *HATE* this change! [side note: The 'from MODULE import ...' form is evil and I have abandoned its use in favor of the 'import MODULE' form in 1987 or so, as our Modula-2 programs got bigger and bigger. With 20+ software developers working on a ~1,000,000 LOC of Modula-2 software system, this decision proofed itself well. The situation with Python is comparable. Avoiding 'from ... import' rewards itself later, when your software has grown bigger and when it comes to maintaince by people not familar with the used modules. ] May be I didn't understand what this new subdivision of the standard library should achieve. The library documentation provides a existing logical subdivision into chapters, which group the library into several kinds of services. IMO this subdivision could be discussed and possibly revised. But at the moment I got the impression, that it was simply ignored. Why? What's so bad with it? Why is a subdivision on the documentation level not sufficient? Why should modules be moved into packages? I don't get it. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)

Hi, Peter. Your question as to the purpose of module reorganization is well worth asking, and perhaps we should stand back for a while and try to really answer it well first. I think that my answers for your question would be: 1. To alleviate potential namespace collision. 2. To permit talking about packages as a unit. I hereby solicit other reasons from the rest of the group... Reason #1 is not a serious problem yet, but i think i've seen a few cases where it might start to be an issue. Reason #2 has to do with things like assigning people responsibility for taking care of a particular package, or making commitments about which packages will be available with which distributions or platforms. Hence, for example, the idea of the "unix" package. Neither of these reasons necessitate a deep and holy hierarchy, so we certainly want to keep it shallow and simple if we're going to do this at all.

If the result of this renaming initiative will be that I can't use import sys, os, time, re, struct, cPickle, parser import Tkinter; Tk=Tkinter; del Tkinter anymore in Python 1.x and instead I have to change this into (for example): form posix import time from text import re from bin import struct from Python import parser from ui import Tkinter; ...

Won't import sys, os, time.time, text.re, bin.struct, data.pickle, python.parser also work? ...i hope?

The library documentation provides a existing logical subdivision into chapters, which group the library into several kinds of services. IMO this subdivision could be discussed and possibly revised. But at the moment I got the impression, that it was simply ignored. Why? What's so bad with it?

I did look at the documentation for some guidance in arranging the modules, though admittedly it didn't direct me much. -- ?!ng "In the sciences, we are now uniquely privileged to sit side by side with the giants on whose shoulders we stand." -- Gerald Holton

...

...

...

...

"PF" == Peter Funk writes:

PF> That is even worse. So not only the 'import' sections, which I PF> usually keep at the top of my modules, have to be changed: This PF> way for example 're.compile(...' has to be changed into PF> 'text.re.compile(...' all over the place possibly breaking the PF> 'Maximum Line Length' styleguide rule. There is nothing wrong with changing only the import statement: from text import re The only problematic use of from ... import ... is from text.re import * which adds an unspecified set of names to the current namespace. Jeremy

On Mon, 27 Mar 2000, Ken Manheimer wrote:

I also thought we had discussed providing transparency in general, at least of the 1.x series. ?

Yes, but it would be clearly marked as deprecated in 1.7, print out error messages in 1.8 and won't work at all in 3000. (That's my view on the point, but I got the feeling this is where the wind is blowing). So the transperancy mechanism is intended only to be "something backwards compatible"...it's not supposed to be a reason why things are ugly (I don't think they are, though). BTW: the transperancy mechanism I suggested was not pushing things into the import path, but rather having toplevel modules which "from import *" from the modules that were moved. E.g., re.py would contain # Deprecated: don't import re, it won't work in future releases from text.re import * -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html http://www.linux.org.il -- we put the penguin in .com

On Tue, 28 Mar 2000, Jeremy Hylton wrote:

...

...

...

...

...

"KLM" == Ken Manheimer writes:

...

...

The only problematic use of from ... import ... is from text.re import * which adds an unspecified set of names to the current namespace.

KLM> The other gotcha i mean applies when the thing you're importing KLM> is a terminal, ie a non-module. Then, changes to the KLM> assignments of the names in the original module aren't KLM> reflected in the names you've imported - they're decoupled from KLM> the namespace of the original module.

This isn't an import issue. Some people simply don't understand that assignment (and import as form of assignment) is name binding. Import binds an imported object to a name in the current namespace. It does not affect bindings in other namespaces, nor should it.

I know that - i was addressing the asserted evilness of from ... import ... and how it applied - and didn't - w.r.t. packages.

KLM> I thought the other problem peter was objecting to, having to KLM> change the import sections in the first place, was going to be KLM> avoided in the 1.x series (if we do this kind of thing) by KLM> inherently extending the import path to include all the KLM> packages, so people need not change their code? Seems like KLM> most of this would be fairly transparent w.r.t. the operation KLM> of existing applications.

I'm not sure if there is consensus on backwards compatibility. I'm not in favor of creating a huge sys.path that includes every package's contents. It would be a big performance hit.

Yes, someone reminded me that the other (better, i think) option is stub modules in the current places that do the "from ... import *" for the right values of "...". py3k finishes the migration by eliminating the stubs. Ken klm@digicool.com

Peter Funk said:

The library documentation provides a existing logical subdivision into chapters, which group the library into several kinds of services. IMO this subdivision could be discussed and possibly revised. But at the moment I got the impression, that it was simply ignored. Why? What's so bad with it?

Ka-Ping Yee writes:

I did look at the documentation for some guidance in arranging the modules, though admittedly it didn't direct me much.

Fred L. Drake, Jr. writes:

The library reference is pretty well disorganized at this point. I want to improve that for the 1.6 docs.

Let me just mention where my inspirations came from: shame of shames, it came from Perl. It's hard to use Perl's organization as is, because it doesn't (view itself) as a general purpose langauge: so things like CGI.pm are toplevel, and regex's are part of the syntax. However, there are a lot of good hints there. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html http://www.linux.org.il -- we put the penguin in .com

Skip Montanaro writes:

Perhaps it makes sense to revise the library reference manual's documentation to reflect the proposed package hierarchy once it becomes concrete.

I'd go for this. ;) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> Corporation for National Research Initiatives

Hi!

Peter Funk wrote:

...

Why should modules be moved into packages? I don't get it.

Fredrik Lundh:

fwiw, neither do I...

Pheeewww... And I thought I'am the only one! ;-)

I'm not so sure that Python really needs a simple reorganization of the existing set of standard library modules. just moving the modules around won't solve the real problems with the 1.5.2 std library...

Right. I propose to leave the namespace flat. I like to argue with Brad J. Cox ---the author of the book "Object Oriented Programming - An Evolutionary Approach" Addison Wesley, 1987--- who proposes the idea of what he calls a "Software-IC": He looks closely to design process of electronic engineers which ussually deal with large data books with prefabricated components. There are often hundreds of them in such a databook and most of them have terse and not very mnemonic names. But the engineers using them all day *know* after a short while that a 7400 chip is a TTL-chip containing 4 NAND gates. Nearly the same holds true for software engineers using Software-IC like 're' or 'struct' as their daily building blocks. A software engineer who is already familar with his/her building blocks has absolutely no advantage from a deeply nested namespace. Now for something completely different: Fredrik Lundh about the library documentation:

here's one proposal: http://www.pythonware.com/people/fredrik/librarybook-contents.htm

Whether 'md5', 'getpass' and 'traceback' fit into a category 'Commonly Used Modules' is ....ummmm.... at least a bit questionable. But we should really focus the discussion on the structure of the documentation. Since many standard library modules belong into several logical catagories at once, a true tree structured organization is simply not sufficient to describe everything. So it is important to set up pointers between related functionality. For example 'string.replace' is somewhat related to 're.sub' or 'getpass' is related to 'crypt', however 'crypt' is related to 'md5' and so on. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)

Maybe we should start a separate list to discuss a package hierarchy for 1.7. But for 1.6, forget it.

Yes! Please! --Guido van Rossum (home page: http://www.python.org/~guido/)

Andrew M. Kuchling wrote: [snip]

2) Right now there's no way for third-party extensions to add themselves to a package in the standard library. Once Python finds foo/__init__.py, it won't look for site-packages/foo/__init__.py, so if you grab, say, "crypto" as a package name in the standard library, it's forever lost to third-party extensions.

That way lies madness. While I'm happy to carp at Java for requiring "com", "net" or whatever as a top level name, their intent is correct: the names grabbed by the Python standard packages belong to no one but the Python standard packages. If you *don't* do that, upgrades are an absolute nightmare. Marc-Andre grabbed "mx". If (as I rather suspect <wink>) he wants to remake the entire standard lib in his image, he's welcome to - *under* mx. What would happen if he (and everyone else) installed themselves *into* my core packages, then I decided I didn't want his stuff? More than likely I'd have to scrub the damn installation and start all over again. - Gordon

Moshe Zadka wrote:

On Tue, 28 Mar 2000, Gordon McMillan wrote:

...

What would happen if he (and everyone else) installed themselves *into* my core packages, then I decided I didn't want his stuff? More than likely I'd have to scrub the damn installation and start all over again.

I think Greg Stein answered that objection, by reminding us that the filesystem isn't the only way to set up a package hierarchy.

You mean when Greg said:

Assuming that you use an archive like those found in my "small" distro or Gordon's distro, then this is no problem. The archive simply recognizes and maps "text.encoding.macbinary" to its own module.

I don't know what this has to do with it. When we get around to the 'macbinary' part, we have already established that 'text.encoding' is the parent which should supply 'macbinary'.

In particular, even with Python's current module system, there is no need to scrub installations: Python core modules go (under UNIX) in /usr/local/lib/python1.5, and 3rd party modules go in /usr/local/lib/python1.5/site-packages.

And if there's a /usr/local/lib/python1.5/text/encoding, there's no way that /usr/local/lib/python1.5/site- packages/text/encoding will get searched. I believe you could hack up an importer that did allow this, and I think you'd be 100% certifiable if you did. Just look at the surprise factor. Hacking stuff into another package is just as evil as math.pi = 42.

Anyway, I already expressed my preference of the Perl way, over the Java way. For one thing, I don't want to have to register a domain just so I could distribute Python code <wink>

I haven't the foggiest what the "Perl way" is; I wouldn't be surprised if it relied on un-Pythonic sociological factors. I already said the Java mechanics are silly; uniqueness is what matters. When Python packages start selling in the four and five figure range <snort>, then a registry mechanism will likely be necessary. - Gordon

On Wed, 29 Mar 2000, Gordon McMillan wrote:

Moshe Zadka wrote:

...

On Tue, 28 Mar 2000, Gordon McMillan wrote:

...

What would happen if he (and everyone else) installed themselves *into* my core packages, then I decided I didn't want his stuff? More than likely I'd have to scrub the damn installation and start all over again.

I think Greg Stein answered that objection, by reminding us that the filesystem isn't the only way to set up a package hierarchy.

You mean when Greg said:

...

Assuming that you use an archive like those found in my "small" distro or Gordon's distro, then this is no problem. The archive simply recognizes and maps "text.encoding.macbinary" to its own module.

I don't know what this has to do with it. When we get around to the 'macbinary' part, we have already established that 'text.encoding' is the parent which should supply 'macbinary'.

good point...

...

In particular, even with Python's current module system, there is no need to scrub installations: Python core modules go (under UNIX) in /usr/local/lib/python1.5, and 3rd party modules go in /usr/local/lib/python1.5/site-packages.

And if there's a /usr/local/lib/python1.5/text/encoding, there's no way that /usr/local/lib/python1.5/site- packages/text/encoding will get searched.

I believe you could hack up an importer that did allow this, and I think you'd be 100% certifiable if you did. Just look at the surprise factor.

Hacking stuff into another package is just as evil as math.pi = 42.

Not if the package was designed for it. For a "package" like "net", it would be perfectly acceptable to allow third-parties to define that as their installation point. And yes, assume there is an importer that looks into the installed archives for modules. In the example, the harder part is determining where the "text.encoding" package is loaded from. And yah: it may be difficult to arrange the the text.encoding's importer to allow for archive searching. Cheers, -g -- Greg Stein, http://www.lyra.org/

...

Marc-Andre grabbed "mx". If (as I rather suspect <wink>) he wants to remake the entire standard lib in his image, he's welcome to - *under* mx.

Right, that's the way I see it too. BTW, where can I register the "mx" top-level package name ? Should these be registered in the NIST registry ? Will the names registered there be honored ?

I think the NIST registry is a failed experiment -- too cumbersome to maintain or consult. We can do this the same way as common law handles trade marks: if you have used it as your brand name long enough, even if you didn't register, someone else cannot grab it away from you.

...

What would happen if he (and everyone else) installed themselves *into* my core packages, then I decided I didn't want his stuff? More than likely I'd have to scrub the damn installation and start all over again.

That's a no-no, IMHO. Unless explicitly allowed, packages should *not* install themselves as subpackages to other existing top-level packages. If they do, its their problem if the hierarchy changes...

Agreed. Although some people seem to *want* this. Probably because it's okay to do that in Java and (apparently?) in Perl. And C++, probably. It all probably stems back to Lisp. I admit that I didn't see this subtlety when I designed Python's package architecture. It's too late to change (e.g. because of __init__.py). Is it a problem though? Let's be open-minded about this and think about whether we want to allow this or not, and why... --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 28 Mar 2000, Fredrik Lundh wrote:

...

IMO this subdivision could be discussed and possibly revised.

here's one proposal: http://www.pythonware.com/people/fredrik/librarybook-contents.htm

Wow. I don't think i hardly ever use any of the modules in your "Commonly Used Modules" category. Except traceback, from time to time, but that's really the only one! Hmm. I'd arrange things a little differently, though i do like the category for Data Representation (it should probably go next to Data Storage though). I would prefer a separate group for interpreter-and-development-related things. The "File Formats" group seems weak... to me, its contents would better belong in a "parsing" or "text processing" classification. urlparse definitely goes with urllib. These comments are kind of random, i know... maybe i'll try putting together another grouping if i have any time. -- ?!ng