Mailman 3 September 1999 - Python-Dev

Alternative Approach to Relative Imports
by M.-A. Lemburg 04 Oct '99

04 Oct '99

Hi everybody, I'm currently busy converting my code to use mx.DateTime instead of just DateTime and must say that this is a real pain. Now instead of reviving the __ discussion I'd like to turn to what could be a workable compromise (I think Jim proposed something along these lines already and there was similar code in ni.py). The proposal is really only an addition to the lookup scheme used by the importer. No additional syntax is involved and best of all, it is backward compatible... The current lookup does the following if you want to import a module E from module A.B.C.D: 1. check A.B.C.E 2. check E 3. fail Now instead of failing we could add a lookup method that walks up the package structure: 3. check A.B.E 4. check A.E [5. check E -- already done] 6. fail so that the complete scheme looks like this: 1. check A.B.C.E 2. check E 3. check A.B.E 4. check A.E [5. check E -- already done] 6. fail That way I could leave intra-mx-package imports untouched and still have the convenience of achieving the goal of making my mx subpackages work in the mx context *plus* in the top-level context thus allowing a backward compatible and flexible setup for mx* users. Note that the scheme finds exactly the same modules it did previously, plus perhaps some more (which is intended), and it does not involve any search path hacks. How is that for a compromise ? [Ducking for cover ;-)] -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 100 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

8 16

Naming and Binding of Objects (was: Relative Package Imports)
by Vladimir Marangozov 03 Oct '99

03 Oct '99

The relative import thread has hit me with full force. I'm not sure I understand all of what has been said, but since I know this is mainly a naming issue, I want to point you to one of the best CS papers I have had in my hands. It's not really my style to recommend references, but this one should be of major interest to this audience (and Python respectively): Saltzer J., "Naming and Binding of Objects", in Bayer R., "Operating Systems - An Advanced Course", pp. 99-208, LNCS 60, 1978. It's not available online (it was written on a typewriter), so I'd be happy to send a hard copy of it to anyone who raises a hand in private mail (or cannot find LNCS 60). It's simply a jewel. This guy has understood everything (and I suspect Guido has read this one before implementing Python ;-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

3 2

getopt helper?
by Mark Hammond 03 Oct '99

03 Oct '99

A little thought I have had a few times now, so it is time to pass it on for comment. Basically _every_ time I use getopt, I write code like this: import getopt try: opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:") except getopt.error, why: print why print usage % (os.path.basename(sys.argv[0],)) sys.exit(1) Every single time. I have never used getopt without this code. How about we put this functionality into getopt itself? It could be triggered either by adding a new "usage" param defaulting to None, or by adding a new entry point. ie: opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:", usage=usage) or opts, args = getopt.getoptex(sys.argv[1:], "slc:d:e:", usage=usage) I know it is fairly trivial, but IMO is such a useful module and the pattern is used so regularly that is seems to make sense to add it. Any thoughts? If it is seen favourably, how should we spell it? Mark.

8 15

Re: [Python-Dev] Naming and Binding of Objects (was: Relative Package Imports) (fwd)
by David Ascher 30 Sep '99

30 Sep '99

---------- Forwarded message ---------- On Sat, 18 Sep 1999, Vladimir Marangozov wrote: > > Saltzer J., "Naming and Binding of Objects", in Bayer R., "Operating > Systems - An Advanced Course", pp. 99-208, LNCS 60, 1978. > > It's not available online (it was written on a typewriter), so I'd be > happy to send a hard copy of it to anyone who raises a hand in private > mail (or cannot find LNCS 60). I then asked him for a copy stating: > If you send it to me, I can OCR it and make it available online. and he generously sent it to me. The problem is that I hadn't noticed the length of the manuscript. It's over a hundred pages, and the copy is nth generation, making OCR pretty much useless. So, if anyone wants a copy, I can make and send copies (which would make most sense for folks in the US -- sending things from France isn't cheap). So far, it's good reading. Funny to see "file name" in quotes in the text. --david

3 3

Tackling circular dependencies in 2.0?
by Skip Montanaro 22 Sep '99

22 Sep '99

I think in Python 2.0 it would be nice to have some way to reclaim circular dependencies without the programmer explicitly having to do something like implement a destroy() method and requiring other programmers to (remember to) call it. I forget what the current state of affairs is w.r.t. future memory management in Python. Not knowing anything much about memory management, would it be possible to have a sort of mixed ref count/garbage collection system where you only use the gc stuff as a last resort? My thought is that it would be useful to use gc to find and reclaim circular garbage. can-you-tell-I-just-got-bitten-by-a-circular-reference?-ly y'rs Skip Montanaro | http://www.mojam.com/ skip(a)mojam.com | http://www.musi-cal.com/~skip/ 847-971-7098 | Python: Programming the way Guido indented...

5 10

Re: Path hacking
by James C. Ahlstrom 22 Sep '99

22 Sep '99

"Guido van Rossum" wrote: > --> The solution: Ah, finally a specific proposal... > Suppose your application (as a whole, not the individual top-level > script) is called Spam -- this may well also be the name of your > top-level package. Then start each top-level script with the single > line > > import Spam_path > > before importing anything else. This should not be necessary if you use the name "sitecustomize" instead of "Spam_path" right? The file sitecustomize.py is automatically imported. Actually all this sounds like site.py all over again. > Your installer, once it knows the absolute pathname of your > application's root directory, crafts a file Spam_path.py which > contains code that inserts the right absolute pathname into sys.path. I don't think this is necessary either. The sys module is available. So sitecustomize.py can say: import sys mydir = sys.path[0] if not mydir: import os mydir = os.getcwd() sys.path = [mydir] # To be really extreme about it # Note: inserting mydir as sys.path[0] should be redundant but is not > Your installer then installs a copy of this file (or a symbolic link > to it) *in each bin directory where it installs top-level Python > scripts*. > > Because the script's directory is first on the default path, the Spam > scripts will pick up Spam_path without any help from $PYTHONPATH. Hmmm. Is this really true? Nothing else, for example the registry, can change sys.path[0]? Ever? Please say yes. > I know this doesn't resolve the relative import thread (how's that > going by the way? :-) but Barry & Fred & I agree that this is the best > solution to the problem stated in Barry's message to which I am > following up here. This is a good idea, but there are a few problems. It depends on sys.path[0] being the directory of the Python file being executed as the main program. I guess I never really trusted this before. I think if this is the case it should never be ''. A relative path or no path on the command line (the __main__ program) should be replaced by the full path in the sys module setup. Then the "mydir = os.getcwd()" above is not necessary. And inserting mydir as sys.path[0] is truly redundant should the current directory change (as it certainly will). This is currently a problem with sys.path[0] which should be fixed no matter what else happens. The files exceptions.py and site.py must be in all the bin directories as well as sitecustomize.py because they are automatically imported in Py_Initialize(). The above doesn't work when you start the Python command interpreter (no main). I know, its a minor point. It seems to me this totally solves Jim Fulton's and Marc's problem and makes "__" unnecessary. You just install zope and mx in zopedir, perform the above, and presto you have a new private name space where you can control all your names. But there must be some problem here I haven't thought of. I still worry that this is not powerful enough. Greg Stein has volunteered to re-write import.c in Python (using imputil.py) and this is a Great Idea. Lots of Python could probably be written in itself. I would like to try writing the main program in Python and eliminating the special freeze main program. Once you start on this road (and I think it is a good road) you have Python code which is more truly part of the binary interpreter than a library. Proposal: Use a special PYTHONPATH on startup to find "special" Python files which are really part of the interpreter. There are three directories Python knows about. Namely sys.path[0] (once it is fixed), sys.executable and sys.dllfullpath, the directory of python15.dll or other shared library (once it is added to sys). How about prepending the single directory sys.executable to sys.path during Py_Initialize()? And demanding that modules like the new Greg_importer.py[c], exceptions.py[c] and site.py[c] be placed there. Actually I would prefer sys.dllfullpath if it exists, since that is where the interpreter is, and I am trying to associate these special internal Python files exactly with their correct Python interpreter. Alternative Proposal: Py_Initialize() first imports its files from sys.executable + '/' + PyInternal.pyl (again I prefer sys.dllfullpath). PyInternal.pyl is a Python library file (like a Java Jar file) which would contain modules like exceptions, etc. The PyInternal.pyl file has the standard Python library file format (whatever that turns out to be). It is not an error if this file is absent. Jim Ahlstrom

9 27

wet signature form clarification
by Guido van Rossum 20 Sep '99

20 Sep '99

Several of you have asked me a clarification of the "contribution description" required on the wet signature form; given that you are all very prolific contributors with a weak memory, it would be nice if you could write something like "everything I've contributed until Sept. 18, 1999" in the space there rather than dig out all the specific module names. I got a call from CNRI's lawyer today, and she agreed that this was fine. Boy am I pleased today! I think she called it a "group description" or something like that. Anyway, if you were waiting for a ruling on this issue, please send in your signed form! (I'd appreciate it if the signature was dry by the time I got it. :-) The form is at http://python.org/1.5/wetsign.html --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Preview of some distutils news
by Greg Ward 20 Sep '99

20 Sep '99

Judging by the recent traffic on python-dev, lots of people here are interested in the problems of distributing and installing Python modules. So I thought y'all might want to know that I've finally got some interesting news to report on the distutils-sig, and will shortly be reporting it, soliciting feedback, testers, etc. If you're not already on that list, this might be a good time to join. (Oh, the news? Distutils can now compile and install NumPy, PIL, and mxDateTime (just because I happened to have those three distributions sitting around on my home PC). Of course there are all sorts of caveats and limitations, so c'mon over to the distutils-sig if you want to hear what they are and discuss how to fix them. Later this week, of course, after I've checked in all the relevant code.) Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

1 0

Relative Package Imports
by M.-A. Lemburg 20 Sep '99

20 Sep '99

Hi everybody, I've spent the last two hours trying to get relative package imports to work because I need them for my extension packages which will soon all move under a new top-level package name to overcome the conflicts with PIL and Zope. Here are the results... Demo Package Structure: (see the attached demopkg.zip) [a] [b] bc.py ab.py With the attached patch you can do the following: # Pretty useless... import a.b.__.ab # From inside bc.py: from __ import ab # At top-level (also useless, but shows how this situation is handled): import __.sys # __ is bound to None since we are at top-level; sys is still # being loaded though. Of course, common usage will be of the form: form __.__ import submodule_at_higher_level Please tell me what you think and give it a try. It's a first try and may have some design errors. Especially the way head and tail are treated in Python/import.c:import_module_ex may cause trouble -- I need help here. Note: The patch is against the CVS version. If you run Python in verbose mode, the patch will produce some verbose output of what it's doing. Enjoy, -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 113 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ --- /home/lemburg/orig/Python/Python/import.c Fri Apr 9 19:00:51 1999 +++ Python/import.c Fri Sep 10 20:51:02 1999 @@ -1572,10 +1572,14 @@ load_next(mod, altmod, p_name, buf, p_bu char *dot = strchr(name, '.'); int len; char *p; PyObject *result; + if (Py_VerboseFlag) + printf("# load_next: (1) name='%s', buf='%.*s'\n", + name,*p_buflen,buf); + if (dot == NULL) { *p_name = NULL; len = strlen(name); } else { @@ -1586,10 +1590,39 @@ load_next(mod, altmod, p_name, buf, p_bu PyErr_SetString(PyExc_ValueError, "Empty module name"); return NULL; } + /* Handle "__" indicator telling the import mechanism to + continue the search one level higher in the package + hierarchy */ + if (strncmp(name,"__",len) == 0) { + PyObject *modules = PyImport_GetModuleDict(); + + /* Strip the final dotted name from buf */ + dot = strrchr(buf, '.'); + if (dot == NULL) + *p_buflen = 0; + else + *p_buflen = dot - buf; + buf[*p_buflen] = '\0'; + + /* Fetch the parent module or revert to a top-level search */ + if (*p_buflen > 0) { + mod = PyDict_GetItemString(modules,buf); + if (mod == NULL) { + PyErr_SetString(PyExc_SystemError, + "Parent module missing"); + return NULL; + } + } + else + mod = Py_None; + Py_INCREF(mod); + return mod; + } + p = buf + *p_buflen; if (p != buf) *p++ = '.'; if (p+len-buf >= MAXPATHLEN) { PyErr_SetString(PyExc_ValueError, @@ -1597,10 +1630,14 @@ load_next(mod, altmod, p_name, buf, p_bu return NULL; } strncpy(p, name, len); p[len] = '\0'; *p_buflen = p+len-buf; + + if (Py_VerboseFlag) + printf("# load_next: (2) modname='%s', fullname=buf='%s'\n", + p,buf); result = import_submodule(mod, p, buf); if (result == Py_None && altmod != mod) { Py_DECREF(result); /* Here, altmod must be None and mod must not be None */

13 81

License cleanup
by Guido van Rossum 18 Sep '99

18 Sep '99

After more than four years of living with an out-of-dated license for Python, CNRI has finally agreed to clean up Python's copyright status. I expect that this won't have any real effect before Python 1.6 is released, but I am required to start preparing for the transition now. We will use a new license (a clone of the JPython license) and we will require that all contributors explicitly allow us the use of their contribution: either a few email paragraphs in an email message, or a longer form with a wet signature, depending on the size of the contribution. I believe the text of the license and forms we use is quite uncontroversial; these very same words have been used for JPython for quite a while. The words are all on the web: http://www.python.org/1.5/pylicense.html [proposed license] http://www.python.org/1.5/bugrelease.html [email release] http://www.python.org/1.5/wetsign.html [wet signature release] If you are reading python-dev but you never contributed any code to Python, you can stop reading now. If you *did* contribute code to Python, however, I'd love it if you saved me some work and filled out the wet signature form and mailed it to me at the given address. If you need help jogging your memory what your contributions were, send me email; I can try grepping the CVS files for your name. If you believe that special circumstances exist that make it impossible or difficult for you to sign the form, please send me email, and we'll discuss the matter. If you contributed something and I don't hear from you, you will eventually hear from me again -- but I hope I can save myself the hassle of writing each of you through this mass mailing. Thanks in advance! --Guido van Rossum (home page: http://www.python.org/~guido/)

14 30

Python-Dev September 1999