On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:

I. eval() to accept custom mapping arguments for globals and locals. This makes it possible to write a smart __getitem__ method for applications like spreadsheets or case-insensitive evaluation.

http://mail.python.org/pipermail/python-dev/2002-October/029770.html [and the rest of that thread] A 3% slowdown in the common case was considered too big to allow a feature like this to be included. The slowdown might be smaller if the builtin module wasn't allowed to be anything but a dict instance, or might disappear if there were two versions of ceval.c were included, and one used PyObject_ APIs and the other used PyDict_ APIs (as long as only one was fighting for space in the cpu cache, main memory, etc) Jeff

Barry Warsaw writes:

V <wink>. Explicit global imports. I want a way to spell "import this from the global standard library even if there's a this.py in the local directory".

I like the syntax: import global <dotted-name> and from global <dotted-name> import <name> Look Guido, no new keywords! ;-) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation

On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:

I'd much rather invent new syntax to spell *local* imports. I like

from .module import something

You can even generalize and write

from ..module import something

to reference your package's parent package. :-)

Actually, if truth be told, I'd love to just ban local imports. I understand that might be controversial though <wink>. The next best thing would be to force local imports to use special syntax (I was going to say "ugly" :) like what you suggest above, but I'm afraid that that might be more disruptive, requiring future imports and such. I think having a syntax for spelling global imports can be done in a backwards compatible way for 2.4. -Barry

On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:

...

I'd much rather invent new syntax to spell *local* imports. I like

from .module import something

You can even generalize and write

from ..module import something

to reference your package's parent package. :-)

Actually, if truth be told, I'd love to just ban local imports. I understand that might be controversial though <wink>.

Works for me. :-)

The next best thing would be to force local imports to use special syntax (I was going to say "ugly" :) like what you suggest above, but I'm afraid that that might be more disruptive, requiring future imports and such. I think having a syntax for spelling global imports can be done in a backwards compatible way for 2.4.

Well, but since you want all imports to be global, it'd be insane to introduce *new* syntax for global imports, wouldn't it? --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 2003-12-15 at 11:52, Skip Montanaro wrote:

Guido> I'd much rather invent new syntax to spell *local* imports.

Barry> Actually, if truth be told, I'd love to just ban local imports. Barry> I understand that might be controversial though <wink>.

Okay, so what are global and local imports?

Sorry, let me rephrase. I'd love to ban relative imports. Personally, I think all imports should be specified using fully-qualified absolute package paths. -Barry

On Mon, 15 Dec 2003, Guido van Rossum wrote:

...

Barry Warsaw writes:

...

V <wink>. Explicit global imports. I want a way to spell "import this from the global standard library even if there's a this.py in the local directory".

I like the syntax:

import global <dotted-name>

and

from global <dotted-name> import <name>

Look Guido, no new keywords! ;-)

I'd much rather invent new syntax to spell *local* imports. I like

from .module import something

You can even generalize and write

from ..module import something

to reference your package's parent package. :-)

As i've said before (though it's been a while) i think people want this, it would be useful, and i think it would work well. Barry wrote, in response to guido's message: < Actually, if truth be told, I'd love to just ban local imports. I < understand that might be controversial though <wink>. I don't understand the desire to totally prevent relative imports. It would be quite useful in avoiding growing module-naming problems, where you have to be careful about shadowing a global module with one in your package. I *can* understand objecting to the ambiguity in the current situation, where you don't know whether a module import will resolve to a sibling module or one in higher up in the hierarchy. That is why i strongly prefer having a leading dot to explicitly signify a relative import, and originally proposed it that way. I expect that the elimination of the ambiguity would mitigate the objection to relative imports - and we would do away with the shadowing problems, and even have easy relocation of modules, if there are cases where software does need to move around. Ken klm@zope.com

...

Well, but since you want all imports to be global, it'd be insane to introduce *new* syntax for global imports, wouldn't it?

If we banned relative (a.k.a. local) imports, yes definitely.

from __future__ global_imports

I think this ought to be a *global* flag rather than a per-module flag. E.g. after setting sys.allow_relative_import = False all imports anywhere would be interpreted as absolute imports. This would mean you couldn't have some code that still uses relative imports, but the problem with the __future__ statement is that it seems so pointless: packages that only use absolute imports don't need it, and packages that use relative imports break if it is used. About the only time where the __future__ statement would make a difference is when a package defines a local module whose name is the same as that of a global module, *and* the package also wants to import the global module. I would personally solve that by renaming the local module though... --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 2003-12-15 at 13:36, Guido van Rossum wrote:

I think this ought to be a *global* flag rather than a per-module flag. E.g. after setting

sys.allow_relative_import = False

all imports anywhere would be interpreted as absolute imports.

This would mean you couldn't have some code that still uses relative imports, but the problem with the __future__ statement is that it seems so pointless: packages that only use absolute imports don't need it, and packages that use relative imports break if it is used.

You probably want something at the package level. This may not be feasible, but it seems like you want to be able to say, "this package needs to allow relative imports for backwards compatibility". -Barry

At 02:26 PM 12/15/03 -0500, Raymond Hettinger wrote:

...

On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:

...

I. eval() to accept custom mapping arguments for globals and locals. This makes it possible to write a smart __getitem__ method for applications like spreadsheets or case-insensitive evaluation.

http://mail.python.org/pipermail/python-dev/2002-October/029770.html [and the rest of that thread]

[Jeff Epler]

...

A 3% slowdown in the common case was considered too big to allow a feature like this to be included. The slowdown might be smaller if the builtin module wasn't allowed to be anything but a dict instance, or might disappear if there were two versions of ceval.c were included, and one used PyObject_ APIs and the other used PyDict_ APIs (as long as only one was fighting for space in the cpu cache, main memory, etc)

Ordinarily, I would say that 3% is a reasonable price to pay for this functionality but there is that pesky pie throwing event coming up.

There is a workaround for this in CPython, which used to be used by Zope, and which I have used on occasion: scan the code object for LOAD_NAME opcodes, and note what names are used by the code block. Then, look up those names in your non-dictionary dictionary, and copy them into a real dictionary. So, it's got to be something pretty specialized to really need PyObject_* APIs instead of PyDict_* APIs in CPython at present. Why penalize *everything* for such an specialized need?

On Mon, 15 Dec 2003, Guido van Rossum wrote:

...

It would be quite useful in avoiding growing module-naming problems, where you have to be careful about shadowing a global module with one in your package.

I don't see the problem, or I misuderstand what "it" refers to; it seems you have this backwards if it refers to relative imports. Say my package P defines a submodule sys. If we require absolute imports, there is no ambiguity: the submodule sys must be imported as P.sys while the standard sys module can be imported as simply sys.

I guess i think there is a case to be made for relative imports, and it becomes apparent as an enterprise grows. Increasingly at Zope corp we are mixing and matching packages to deliver applications to a customer. This is very encouraging, because it means we are actually getting reuse out of the packages. Currently, we can just plop the things in place, and use them. Without relative imports, we would have to be editing the imports in the packages in each place we use it. This would increase the burden of using the packages and even more of feeding back actual fixes - which we then have to distinguish from what i would see as gratuitous changes to edit import names. If you would grant there's use to avoiding those edits, and so there's use to having relative imports, then you might see a reason to solve the problems where the names in a package conflict with those along the load path. Otherwise, if there's only absolute imports, there's no ambiguity to resolve. I'd say there's a legitimate need for relative imports, and we need some explicit gesture to disambiguate between a relative and absolute import, one way or the other. Ken

On Mon, 2003-12-15 at 14:34, Ken Manheimer wrote:

Currently, we can just plop the things in place, and use them. Without relative imports, we would have to be editing the imports in the packages in each place we use it.

I've heard this before, but I still don't get it, so I'd like to tease this one out more. What does "plop the things in place" mean? Zope 2 has a few standard directories it adds to its PYTHONPATH. If I distribute a 3rd party package Zope wants to use, and that package uses absolute imports, you can just drop the package in one of those directories and it'll just work. Maybe you mean that you want to drop the mythical 3rd party package in somewhere other than a directory on Zope's PYTHONPATH and have it just work. I'd say that's too perverse of a use-case for Python to worry about. One problem with banning relative imports is that Python doesn't allow a top-level package name to live in more than one place on PYTHONPATH. It would be useful for Python to support such things out of the box, say if we wanted to adopt a Java style naming convention for package organizations. Jim uses the term "pure container package" to describe top-level packages that contain no real content in the package except subpackages. "zope" and "zodb" are examples of this in Zope 3. IWBNI Python had better/easier support for weaving together pure container package contents where subdirectories live in different locations on the file system. -Barry

On Mon, Dec 15, 2003 at 02:26:03PM -0500, Raymond Hettinger wrote:

Ordinarily, I would say that 3% is a reasonable price to pay for this functionality but there is that pesky pie throwing event coming up.

Maybe we can add it in right after the benchmarks get run? Jeff

On Mon, 2003-12-15 at 17:49, Bill Janssen wrote:

I'd agree with Ken here. I think the importance of being able to do relative imports grows as well with the ability to compile Python into foreign frameworks, like Jython or .NET.

Why? I'd really like to see some concrete use cases for relative imports, other than just saving some typing. -Barry

Bill Janssen wrote:

It's much better to be able to write (in the new higher-level module bletch.py)

import foo import bar

and have the subpackages under foo and bar continue to work without rewrites, than have to go through all the code under foo and bar and change it to know about the new higher-level bletch namespace.

Right. Or as someone said: Namespaces are one honking great idea -- let's do more of those!

But why do you have to give those packages a different full name? That's the question that I've never seen answered adequately.

I have done this many times, so let me try to describe at least one legitimate usage case. A couple of weeks ago I wrote a software which needs a third party package to work. OTOH, since it's a small package, I don't want to force the user to go after the package, even because I'd have to ensure that my code always work with the available version of that package. Thanks to the relative module importing mechanism, solving that is no harder than copying the third party package into my own package namespace. This idea could probably be expressed in some other way, hacking sys.path or whatever, but I belive this is a fairly common pattern, and I vote for introducing a scheme to differ between local/global importing which would not break the current flexibility. -- Gustavo Niemeyer http://niemeyer.net

On Tue, 16 Dec 2003, Guido van Rossum wrote:

There is the remaining issue of what exactly the syntax would be. I propose to extend the from clause to allow one or more dots before the dotted name, and to make the dotted name optional if at least one leading dot is found. I propose not to change from-less import. Examples:

from .foo import bar from .foo.bar import xxx from . import foobar as barfoo from ..foo.bar import * from ... import foobar, barfoo

Grammar change (see Grammar/Grammar):

dotted_name: NAME ('.' NAME)* | '.'+ [NAME ('.' NAME)*]

On a related and minor note, can we please expand the grammar to allow from...import statements to continue to the next line after a trailing comma? For those of us with large, complex, and componentized projects, it would be very nice. e.g.: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, normalize, row_ops, summary instead of: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary This has been a minor nit that bothers just under the threshold of fixing it myself. ;) Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/

Guido van Rossum wrote:

...

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, normalize, row_ops, summary

instead of:

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary

This has been a minor nit that bothers just under the threshold of fixing it myself. ;)

I guess this could be implemented by allowing [NEWLINE] after the comma in the grammar.

But I'm -0 on this change; I fear that someone accidentally adding a trailing comma (or leaving it in after some rearrangements) will be very puzzled at the syntax error that comes somewhere in the middle of the next statement:

from foo import bar,

bufsize = 12 ^syntax error

you could require parentheses: from OPAL.Modules.Financial.Reports.COGS import ( generate, webOptions, normalize, row_ops, summary ) (after all, from-import is a kind of assignment, and "a, b = c" is the same thing as "(a, b) = c"...) </F>

On 16-dec-03, at 19:26, Guido van Rossum wrote:

There is the remaining issue of what exactly the syntax would be. I propose to extend the from clause to allow one or more dots before the dotted name, and to make the dotted name optional if at least one leading dot is found. I propose not to change from-less import. Examples:

from .foo import bar from .foo.bar import xxx from . import foobar as barfoo from ..foo.bar import * from ... import foobar, barfoo

Is "from . import *" allowed? Whenever I start to think about it I tend to go into oscillation (yes because within package foo its the same as "from foo import *" on the outside; no because "import *" isn't allowed today either). -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

Guido:

...

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary

But I'm -0 on this change; I fear that someone accidentally adding a trailing comma ... will be very puzzled

How about a "suite" of things to import? from OPAL.Modules.Financial.Reports.COGS import: generate, webOptions, normalize row_ops, summary Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

At 02:09 PM 12/17/03 +1300, Greg Ewing wrote:

Guido:

...

...

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary

But I'm -0 on this change; I fear that someone accidentally adding a trailing comma ... will be very puzzled

How about a "suite" of things to import?

from OPAL.Modules.Financial.Reports.COGS import: generate, webOptions, normalize row_ops, summary

First impression: cool! Second impression: "suites" contain statements, not a list. Third impression: why no comma after 'normalize'?

Note that ALL current module references would be type C according to this. Yes, this is a semantic change, but one that I believe would be for the best in the long run, and don't think would cause a lot of upheaval.

Yes, I like this. Bill

In article <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>, Greg Ewing wrote:

So I think we really want *three* kinds of module reference:

A: Explicitly absolute B: Explicitly relative to the current module C: Searched for upwards in the package hierarchy from the current module

(Note that C is a generalisation of the current "ambiguous" references which only look in two places.)

Suggested syntaxes for these:

A: a.b.c. Path ends with a dot

B: .a.b.c Path begins with a dot

C: a Path neither begins nor ends with a dot a.b.c

Is funny punctuation really the right way to spell an important distinction like this? -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science

[Guido]

Nah. But I liked the suggestion seen here earlier of allowing parenthesis:

from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions, normalize, row_ops, summary)

So do I. Something I'm still acutely aware of despite not using Emacs for years is how much trouble it would be to teach python-mode.el about a unique new way to continue a statement. Parens would "just work". Ditto for IDLE, and tokenize.py, etc etc etc etc. I bet tools parsing import statements for *content* (like pychecker, and perhaps pyclbr.py -- the others mentioned are just looking at form) would need modification regardless, though.

On Tue, 2003-12-16 at 20:16, Guido van Rossum wrote:

Nah. But I liked the suggestion seen here earlier of allowing parenthesis:

from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions, normalize, row_ops, summary)

+1 -Barry

Barry Warsaw wrote:

On Mon, 2003-12-15 at 11:52, Skip Montanaro wrote:

...

Sorry, let me rephrase. I'd love to ban relative imports. Personally, I think all imports should be specified using fully-qualified absolute package paths.

-Barry

If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me. To me an "absolute import" or a "fully-qualified absolute package path" sounds like you know the exact directory that the import is occuring from. E.g.: /home/someone/projects/python/workingprogress/currentproject/thisone.py I can see the use for being able to import that, but usually I would prefer to import either from something relative to the current directory, or relative to the python site-lib, or some such. It's also quite nice to not need to bother to exactly specify just which of those I will find it in. (This is what the pythonpath is for.) Now to me an import relative to the current directory is a relative import, and I assume that this is also what is meant by a local import. By allowing files to cluster relative to the code being written one is able to keep related projects and libraries together rather than just having all the code spread out aimlessly in a huge flat directory. I don't see in what what this is inferior. Since you didn't say why you wanted to ban relative imports, I don't understand your reasons. I can't even guess at them. Even if it were for some efficiency condern about hierarchical directories, that would merely say that the compiled modules should be stuffed into a library folder. (Well, more than one...the user needs to have write permission to the libraries that he creates, but it's just as well that he not have write permissions to the system libraries...without taking special steps.)

Nick Coghlan :

A: from __absolute__.dotted.name import foo B: from __relative__.dotted.name import bar C: from __heirarchy__.dotted.name import foobar D: from dotted.name import barfoo

Then Guido's migration plan would apply to the progression of the D semantics from the Python 2.3 model to being a shorthand for the absolute semantics.

And once _that_ has happened, then we could get rid of the '__absolute__' version in the name of TOOWTDI.

That's where I disagree - I think that type C semantics should be the default/least ugly/easiest to get, not type A. But I wouldn't like any of the others to be as ugly as those, either. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:

A lot people have presented a good case for relative imports. Nobody has argued to keep the status quo (where imports are ambiguous about whether they are meant to be absolute or relative). So I suggest that in 2.4, we introduce the leading dot notation for relative import, while still allowing relative import without a leading dot. In 2.5 we can start warning about relative import without a leading dot (although that will undoubtedly get complaints from folks who have code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import.

I'll just note that where the current status quo trips /me/ up most is when I accidentally have a local module with the same name as a global module, and then I write an import statement expecting to get the standard library module, but end up getting the local module. That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups. I'm still catching up on this thread, but I wanted to throw this out there... -Barry

On Wed, 2003-12-17 at 11:55, Charles Hixson wrote:

Barry Warsaw wrote:

...

Sorry, let me rephrase. I'd love to ban relative imports. Personally, I think all imports should be specified using fully-qualified absolute package paths.

If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me.

Nope, because "absolute package paths" are always calculated relative to directories in sys.path when mapping a package path to a file system path. -Barry

[Barry]

... I'll just note that where the current status quo trips /me/ up most is when I accidentally have a local module with the same name as a global module, and then I write an import statement expecting to get the standard library module, but end up getting the local module.

Indeed, this is common on c.l.py and the Tutor lists, with a twist: someone writes a module in their *current* directory, with a name like random.py, and then "Python breaks mysteriously all over the place", because every import of random ends up with something utterly unexpected. For some reason, "random.py" seems most often to be the actual name used in these cases. Possibly related: I remember pissing away a day a year or so ago trying to track down a segfault in a Zope test that happened only when I ran the test on my McLean box. In the end, it turned out that the test in question was (after untangling the usual layers of Zopish indirection ) deliberately trying to import a module that didn't exist, with an oddball name. It just so happened that somebody had whittled down a segfaulting Python program and uploaded it to SourceForge, with the very same oddball name. I had downloaded that program into my Python build directory, and Zope kept provoking its segfault as a side effect of (unintentionally!) importing it. In large part, I've come to loathe imports -- I never have the foggiest idea how they're going to get satisfied in code I didn't write, and especially in code that plays every trick in (and out) of the book to hide what it's really doing with sys.path mutations, import hooks, poking stuff by magic into sys.modules via well-hidden side effects in scattered __init__.py files, ... it makes Java's rigid, wordy, explicit import scheme look very attractive some days. That you can't trick that into doing arbitrarily unexpected things by magic at runtime isn't entirely a bad thing <wink>.

On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:

On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:

...

On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:

...

That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups.

Alternatively, maybe what you want is a way to say "definitely give me the standard library one, no matter what", e.g.

from stdlib import codecs

Interesting. I see the subtle distinction. One searches only the standard library, the other searches every directory on sys.path. I'm not sure I'd have much need to restrict the search to just the standard library. Plus, would that include site-packages?

I should have been clearer. What I'm basically proposing is to make site-package called "stdlib", leaving a default python install with a very uncluttered top-level namespace, thus reducing unexpected name conflicts. Otherwise, this idea doesn't require any modification to import semantics, except for some magic to make stdlib the default root for imports for backwards compatibility. Thinking about it a bit harder, though, I'm not sure it's as easy as I initially thought, because this would then make importing 3rd party packages not work, i.e. currently you can do: import urllib import pygtk without problems, because they are both top-level. If urllib moved into a "stdlib" package, it's hard to imagine a clean way where old code that does those two lines would still work. I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0. -Andrew.

Guido:

Hmm, maybe the current directory should be last? That way you can import your own modules except if they clash with the standard library. Hm, but there are so many standard library modules that it might still cause frequent complaints from mystified beginners.

Maybe in Python 3.0 the current directory should appear as a package with some standard name. Then applications could use relative imports to import their own modules, and absolute ones to import standard modules, without fear of conflict. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

On Wed, 2003-12-17 at 23:55, Guido van Rossum wrote:

This can be interpreted in two ways though. You could be unfortunate enough to create a module in your package that has the same name as a standard module you also need to import from inside the same package -- that's currently impossible.

Right, and I'd like to make that possible. The least disruptive way I see of doing that is to introduce a magic root package from which things can be imported from. E.g. import __root__.re

But the most common variant I've seen is what Tim Peters describes: having a module in the current directory that accidentally hides a standard module.

Yep, I've been bit by that too.

The latter case does not involve packages and will not be fixed by any import syntax -- it can only be fixed by changing the default sys.path *not* to include the current directory, just like over the last decade or so Unix shells have stopped putting '.' in $PATH.

Yep. I haven't had '.' in my $PATH for a bazillion years. :)

But that has other downsides: it would be a lot harder to begin writing a program broken up into modules. Hmm, maybe the current directory should be last? That way you can import your own modules except if they clash with the standard library. Hm, but there are so many standard library modules that it might still cause frequent complaints from mystified beginners.

I'm beginning to agree (uselessly) with Tim: import itself is a problem. :-)

There ya go! That's thinking outside the box: let's deprecate import altogether :) -Barry

[Guido]

... I'm beginning to agree (uselessly) with Tim: import itself is a problem. :-)

Not useless at all -- that's bonding! Technically useless but humanly essential <wink>. It's no mystery *why* import is "a problem", but I'm not sure insight implies a cure: it's one of the very few places in Python where you can't explicitly say what you intend. Even in an inheritance hierarchy from hell, if I really don't want to chance that please.hit_me() will get satisfied from a class other than BlackJack, I can force it easily -- BlackJack.hit_me(please) There aren't any names to spell "the roots" of all the places Python may search to satisfy an import, so no way either to be explicit about intent. In contrast, nobody can mistake what import java.util.Random; intends. It's not a deep thing, it's just being able to name the intended context. I confess I'm keener on that than on growing more ways to spell more kinds of searches.

Tim Peters wrote:

In contrast, nobody can mistake what

import java.util.Random;

intends. It's not a deep thing, it's just being able to name the intended context. I confess I'm keener on that than on growing more ways to spell more kinds of searches.

I *strongly* agree!!! --- Paolo Invernizzi

Andrew Bennetts writes:

I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0.

Guido tends to shoot people who suggest this, watch out :-) Cheers, mwh -- Lisp nearing the age of 50 is the most modern language out there. GC, dynamic, reflective, the best OO model extant including GFs, procedural macros, and the only thing old-fashioned about it is that it is compiled and fast. -- Kenny Tilton, comp.lang.python

Skip Montanaro writes:

Andrew> Or it can be fixed by not having the entire standard library at Andrew> the toplevel, but instead inside a single "stdlib" package. ... Andrew> The backwards compatibility problems probably means this isn't Andrew> feasible until Python 3.0, though :(

Not necessarily. It has been proposed (more than once I think) that such a stdlib package could be created in parallel using symlinks. At its simplest, I think all you'd need would be a stdlib directory, a nearly empty __init__.py file and symlinks to all the current standard modules and packages. (The __init__.py file would take care of importing builtin modules.)

Are you sure that you won't get multiple copies of the same module floating around? E.g. will you have from httplib import HTTPConnection as HC1 from std.httplib import HTTPConnection as HC2 HC1 is HC2 I think in the scheme sketched above this will be false, which kills the idea stone dead. I might be wrong, though. Cheers, mwh -- Counting lines is probably a good idea if you want to print it out and are short on paper, but I fail to see the purpose otherwise. -- Erik Naggum, comp.lang.lisp

Barry Warsaw writes:

On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:

...

A lot people have presented a good case for relative imports. Nobody has argued to keep the status quo (where imports are ambiguous about whether they are meant to be absolute or relative). So I suggest that in 2.4, we introduce the leading dot notation for relative import, while still allowing relative import without a leading dot. In 2.5 we can start warning about relative import without a leading dot (although that will undoubtedly get complaints from folks who have code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import.

I'll just note that where the current status quo trips /me/ up most is when I accidentally have a local module with the same name as a global module, and then I write an import statement expecting to get the standard library module, but end up getting the local module.

That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups.

That issue I can understand. And I agree there should be a way to state it explicitly. One of the issues here is that this area is a bit under-documented (I know, I should read the source, but I don't have the time right now). At the moment, there are two cases: 1. Import from sys.path. This is what is being called an "absolute" import, and is nice and easy to understand. The key issue is that there is no way to *force* this interpretation in the face of option (2) below. 2. Import from "the package". This is the under-documented bit, but if I understand it correctly, it's basically that from within a module contained in a package, sys.path is conceptually *extended* to include the package's __path__ (which by default contains the directory of the package, but which can be user-modified). Now the big problem here is that behaviour (2) is useful. Simple "relative" imports of one module within a package from another module in the same package are common. Guido's (IMHO ugly) "dot" syntax handles that, by making users explicitly request option (2), and making the current import syntax *only* mean (1). But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__ (heck, it would have made thinking about PEP 302 easier, if nothing else) but (a) it would need a deprecation warning, and (b) Guido himself offered a use case in http://www.python.org/doc/essays/packages.html. This definitely needs a PEP. If we're removing support for __path__, the implications need to be thought through (PEP 302, the pkgutil module, etc etc). If we're not, none of the proposals so far have covered how __path__ gets supported in future. A much simpler proposal, just providing an explicit way of saying "Import from sys.path *only*" may be OK without a PEP. But even then, I'd suspect we should have a PEP explaining *why* it has to be this simple. Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-) Paul. -- This signature intentionally left blank

Paolo> Symlink is one of the things that I would like to have on my XP Paolo> box... As Michael Hudson suggests, symlinks probably won't work anyway. I have something which looks like a symlink on my Win2k box. Windows calls it a Shortcut in listings. Is that not roughly the same thing as a symlink? Skip

[Barry]

...

That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups.

[Paul Moore]

That issue I can understand. And I agree there should be a way to state it explicitly.

One of the issues here is that this area is a bit under-documented (I know, I should read the source, but I don't have the time right now).

At the moment, there are two cases:

1. Import from sys.path. This is what is being called an "absolute" import, and is nice and easy to understand. The key issue is that there is no way to *force* this interpretation in the face of option (2) below.

Right.

2. Import from "the package". This is the under-documented bit, but if I understand it correctly, it's basically that from within a module contained in a package, sys.path is conceptually *extended* to include the package's __path__ (which by default contains the directory of the package, but which can be user-modified).

That's not a very useful way to think about it; when the module is found on __path__ something very *different* happens than when it is found on sys.path. The difference is noticeable if you ask the imported module for its __name__. If you the import was satisfied from __path__, then __name__ will include the package name. If it was satisfied from sys.path, it won't. And __name__ is related to module identity: all modules are stored in sys.modules using their __name__ as the key, so multiple imports of a module with the same __name__ end up referring to the same module (and only the first import causes the module's code to be executed).

Now the big problem here is that behaviour (2) is useful. Simple "relative" imports of one module within a package from another module in the same package are common. Guido's (IMHO ugly) "dot" syntax handles that, by making users explicitly request option (2), and making the current import syntax *only* mean (1).

Actually, what you consider useful is considered harmless by others. Many large packages, including Zope, have adopted a rule of always using absolute imports, to clarify what kind of import is being used.

But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__ (heck, it would have made thinking about PEP 302 easier, if nothing else) but (a) it would need a deprecation warning, and (b) Guido himself offered a use case in http://www.python.org/doc/essays/packages.html.

__path__ has no bearing on the proposals, it is used for relative imports. I think your confusing about how the current import works caused you to think it is relevant.

This definitely needs a PEP. If we're removing support for __path__, the implications need to be thought through (PEP 302, the pkgutil module, etc etc). If we're not, none of the proposals so far have covered how __path__ gets supported in future.

We're not, but they have (by not mentioning it); but I agree that a PEP is needed.

A much simpler proposal, just providing an explicit way of saying "Import from sys.path *only*" may be OK without a PEP. But even then, I'd suspect we should have a PEP explaining *why* it has to be this simple.

This was where this thread started: someone proposed a way of spelling "import from sys.path only". My big problem with that is that *most* imports are intended to be satisfied by sys.path only, so this spelling should be the default, meaning the current spelling.

Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-)

Alas, I have no time. --Guido van Rossum (home page: http://www.python.org/~guido/)

Paul Moore :

But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__

I'll be worried if __path__ is likely to disappear as a result of all this. I'm using it in what will be the next version of my Python GUI library, to switch in different implementations of submodules depending on the platform. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

On Thu, Dec 18, 2003 at 02:16:45PM +1100, Andrew Bennetts wrote:

On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:

...

On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:

...

On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:

...

That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups.

Alternatively, maybe what you want is a way to say "definitely give me the standard library one, no matter what", e.g.

from stdlib import codecs

Interesting. I see the subtle distinction. One searches only the standard library, the other searches every directory on sys.path. I'm not sure I'd have much need to restrict the search to just the standard library. Plus, would that include site-packages?

I should have been clearer. What I'm basically proposing is to make site-package called "stdlib", leaving a default python install with a very uncluttered top-level namespace, thus reducing unexpected name conflicts.

I'm at least a little confused with what people want, as far as I can tell import semantics have three things * what module to import * what name to give that import * where to look for it The 'from' keyword seems to be getting overloaded to mean both what module and where to find it, which the 'stdlib' tries to resolve. Most of the discussion has been about how to improve 'where to look' I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW searching could be 'asbolute' 'relative' or any of the other suggested words. If you wanted to get fancy it could be a list, if it isn't a list people who truly care could cascade try/except on ImportError. -jackdied ps, I like the second version better but it is less like the current version, which is a drawback.

[Jack Diederich]

...

... I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW

searching could be 'asbolute' 'relative' or any of the other suggested words. If you wanted to get fancy it could be a list, if it isn't a list people who truly care could cascade try/except on ImportError.

[Barry Warsaw]

I want to think about that more, but at first glance, it has some appeal. Neat!

Ditto. I think Jack's on to something there. I particularly like the second form, as it's an ongoing irritation to me that some imports begin with "from" and others with "import". In the current "from" form, the most important thing to me is almost never the module I'm importing the thing from, but (of course) the thing I'm importing. Going from import math to import sin from math (or vice versa) is also an easier edit than from import math to from math import sin (because the former doesn't require transposition). Adding a distinct clause to influence search details seems downright Pythonic. Maybe *too* Pythonic <wink>.

Jack Diederich wrote:

I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW

This would work really well for any uses cases I can imagine having. It also has the following benefits: 1. No magic punctuation 2. People who don't need it, just never use a 'searching' clause 3. People encountering it for the first time have a clear flag letting them know something fancy is going on 4. These people have a word they can look up in the Python docs I hadn't even thought of point 4 prior to considering Jack's proposal. If I encounter "import .fred.mary" in some python I didn't write, how the heck do I find out what it means? [...]

ps, I like the second version better but it is less like the current version, which is a drawback.

To head a little further down this sidetrack, the from version of 'import' has puzzled me for a while: A. from mypkg import foo B. import mypkg.foo as foo What, if any, are the semantic differences between A & B? Is it only that A2 works, and B2 doesn't? A2. from mymodule import * B2. import mymodule.* Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268

On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:

...

I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW

searching could be 'absolute' 'relative' or any of the other suggested words. If you wanted to get fancy it could be a list, if it isn't a list people who truly care could cascade try/except on ImportError.

Unfortunately it looks like 'searching' would have to become a reserved word; if we use the same trick we used to avoid making 'as' a reserved word, that would be too confusing to decode in the code generator. The HOW could be just an identifier then.

Too bad the decorator PEP (PEP 318) seems to favor 'as' in the syntax, import foo [relative] is more explicit than the dot syntax and has a nice visual seperator while avoiding the wordiness problem of 'searching'

I'm not sure how well it'll read in actual examples. Let's try. Adapting an example from Opal that was used here before...

from COGS import generate searching relative

To me, that looks a lot like someone left the commas out of

from COGS import generate, searching, relative

moving the 'import' to always be first would alleviate this a bit because 'searching' would always follow a non-list. import generate from COGS searching relative or if we decorate import generate from COGS [relative] from COGS import generate [relative]

In general I'm not a big fan of "wordy" clauses like this -- they work better in languages like SQL or COBOL because those are case-sensitive so you can use a difference in capitalization there [snip] while in Python the long sequence of lowercase words becomes a blur.

as above, moving import to be first helps but doesn't eliminate the wordiness problem[*]. 'as' and 'from' are short enough to be easilly seen as keywords, 'find' or 'look' reads less like english but would be easier to pick out visually than 'searching'

So if the two alternatives are simply 'searching absolute' and 'searching relative'

Did someone mention 'searching scan' in dusty corner somewhere? One drawback to the dot syntax is if there can be more than two ways to do the search. Adding a third way is easier w/ the searching clause instead of adding a 'bang' syntax to the 'dot' syntax . The decorator would do one better by being a placeholder for anything that modifies the import.

A separate suggestion is to switch from "from X import Y" to "import Y from X". This seems a gratuitous change even if it reads or edits marginally better. Now's not the time to tinker with marginal stuff -- that should've been done 10 years ago.

Damn, I really liked that part. I don't get tripped up on from X import Y anymore, but I did when I started w/ python. benign-user-for-life-ly, -jackdied [*] I use an emacs hack to display 'lambda' as 'L' to avoid the same keyword-reads-like-a-variable effect.

On Sunday 21 December 2003 10:52 pm, Jack Diederich wrote:

On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:

...

...

I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW import NAMES as RENAME from MODULE searching HOW

I like Jack's original idea and I've been chewing on it for a few days. His was: import NAMES as RENAME from MODULE searching HOW My humble suggestion: import NAMES import NAMES in MODULE [in HOW] import NAME, OTHER in FOO in RELATIVE import NAME as RENAME in MODULE in ABSOLUTE Playing with it: import sys import path, stat in os import AClass, AValue in AModule in __search__ import AType in Package.Library in __absolute__ It emphasizes the name that's imported and it reduces+reuses a keyword. Reusing "in" is a stretch, of course, and it reading it twice in the same statement might be confusing for some. I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :) Only-a-random-two-cents-ly yours, -- Troy Melhase, troy@gci.net -- Merry Christmas!

On Mon, 2003-12-22 at 04:03, Troy Melhase wrote:

I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :)

I don't either, although my biggest problem with it is that it isn't clear just by looking what the punctuation actually means. I just fear this will be a permanent head scratcher. Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game: import foo by absolute import foo.bar by relative as baz import foo by searching as bar import foo by absolute from bar import foo as bar by absolute from baz IOW, 'by' is a reserved word in the import statement context, and it specifies how to search for modules. The question then is whether 'absolute', 'relative', and 'searching' are also reserved words or built-ins and how those names are equated with the module location algorithms. -Barry

On Mon, 2003-12-22 at 10:36, Guido van Rossum wrote:

...

I don't either, although my biggest problem with it is that it isn't clear just by looking what the punctuation actually means. I just fear this will be a permanent head scratcher.

This argument by itself has very little value. The question isn't whether it's self-describing (very few features beyond decimal integers are); the question is whether it's easy to remember. There are two subclasses of that: easy to remember what it means when you next see it, and easy to remember how to spell it when you next need it. IMO the dot syntax does fine here.

Sorry, it was a pre-coffee argument. To me, the dot-syntax doesn't really tell me whether it forces a relative search or an absolute search. The dot tells me that something different is happening, so maybe when absolute is the default, it tells me there's a switch to relative searching going on. I'm not sure though, so I do think it may potentially fail the "easy to remember what it means when you next see it" test. OTOH, it would be nice to see a Greg Wilson-style newbie poll.

...

Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game:

There have been so many alternatives proposed this morning that I have to spend some time comparing them all...

Hopefully, this stuff will make it into the PEP so you don't have to troll through suggestions buried in endless email threads. -Barry

"Josiah Carlson" wrote in message news:20031222000819.10EC.JCARLSON@uci.edu...

Is None so weird? All I am proposing is a symmetric value to None.

To me that would be All of AllType. Some is in between. Can you think of any uses for All other than being max element? TJR

...

Hopefully, this stuff will make it into the PEP so you don't have to troll through suggestions buried in endless email threads.

That's the plan; I expect that people will review the PEP to make sure their pet proposal is listed.

Thanks -- I'll stop tracking this thread closely then until I see the PEP. --Guido van Rossum (home page: http://www.python.org/~guido/)

"Josiah Carlson" wrote in message news:20031222000819.10EC.JCARLSON@uci.edu...

...

Is None so weird? All I am proposing is a symmetric value to None.

To me that would be All of AllType. Some is in between. Can you think of any uses for All other than being max element?

TJR

Not really. However, I was thinking of Some in the context of graph and dynamic programming algorithms. When programming some algorithms in Python, you need to either pick some large constant (that you hope will be big enough), or have a test to see if a value is None. That is ugly and very not Pythonic. I do like All and AllType though, which gives a better name, as suggested by Guido. - Josiah

Guido van Rossum writes:

...

I don't either, although my biggest problem with it is that it isn't clear just by looking what the punctuation actually means. I just fear this will be a permanent head scratcher.

This argument by itself has very little value. The question isn't whether it's self-describing (very few features beyond decimal integers are); the question is whether it's easy to remember. There are two subclasses of that: easy to remember what it means when you next see it, and easy to remember how to spell it when you next need it. IMO the dot syntax does fine here.

The biggest "clarity" issue I have with the dot notation is that there's no obvious keyword to look up when you want to find out what's going on. The best you could do is look up the "import" section of the manual. Heaven help you if you want to use Google. (Has anyone tried to get any meaningful information from Google on Microsoft's two technologies, COM and .NET? You'd almost think they'd done it deliberately... Seriously, the "what do I search for?" factor bugs me every time I hit it. Paul. -- This signature intentionally left blank

[Paul Moore]

... The fact that [None] compares less than every other value is to me incidental, and arbitrary.

Arbitrary?! I clearly recall when the decision was made that None would compare less than anything else. I was in the same room with Guido, and this is a verbatim transcript of the debate: "As long as we're changing everything here anyway, maybe comparing with None should yield a consistent result." "Like what, maybe smaller than anything else?" "Sure. Or larger." "Ya, one of those. It's not worth it if it just ends up in the middle somewhere." "What does Python do now?" ... [head-scratching] ... "So what did it do before?" ... [more head-scratching] ... "Well, that's no help, then." "The Reference Manual doesn't promise anything about how None will compare, just the 'consistent total ordering' bit". "So that doesn't constrain us." "No, it's neutral." "So maybe it should just be smaller than everything else." "Sure. Or larger." "If it were smaller, then when you sorted a mixed-type list, all the Nones would float to the start." "Good point. If it were larger, they'd float to the end." "I wouldn't like that much." "Me neither." "OK, it's settled." Two lines of code later, the Only Objectively Correct result was achieved. history-is-more-impressive-in-textbooks-ly y'rs - tim

...

[Paul Moore]

...

... The fact that [None] compares less than every other value is to me incidental, and arbitrary.

Arbitrary?! I clearly recall when the decision was made that None would compare less than anything else. I was in the same room with Guido, and this is a verbatim transcript of the debate: <snip> Two lines of code later, the Only Objectively Correct result was achieved.

history-is-more-impressive-in-textbooks-ly y'rs - tim

How about "everything" or "all" as the obverse of "none", ordered higher than all other values? "Some" is far too indeterminate.

Dave LeBlanc Seattle, WA USA

All seems like a reasonable spelling for an object with the given behavior. I started with the name Some because the name made me smile. - Josiah

On Mon, 22 Dec 2003, Devin wrote:

The first part stems from Guido's idea about using a '.' at the beginning of a relative import. It seemed a bit weird at first, but I eventually became very comfortable with the idea after thinking about it in respect to absolute imports. I'll explain what I mean.

If you currently want to import a module 'a.b.c.d', you write:

import a.b.c.d

Now, if you're in the module 'a.b', you could still write:

import a.b.c.d

... but the 'a.b' part is redundant because you're already in the module 'a.b'. If you take away 'a.b', then you end up with:

import .c.d

I also look at it this way, and find it both intuitive and mnemonic. My model is slighly different than yours, though, in a way that simplifies the expression for going up the relative chain. I'll explain...

This flowed well in my head, and looks natural. However, the suggestion about '.' or '..' referring to the parent looks _ugly and unnatural_.

Let's say '.' was used to spell "parent module", and let's assume that the module 'a.b.c.d' wants to import 'a.b.e'. The absolute import would be spelled:

import a.b.e

... and the relative import would be spelled:

import .....e # .[parent].[parent].e

Yuck! (no offense :)

Truly, yuck. But in my model, the leading '.' dot, itself, stands for the containing package, and '..' stands for the containing package's package, and so forth: import ..e # [parent-of-parent].e I don't think it's yucky at all, this way. (I also don't think it's un-pythonic - python uses punctuation all over to denote type literals ('{'/'}'), cues which delineate statement suites (':'), and so forth.) In fact, i think it's a lot more mnemonic and encompassing than the alternatives that use words (though i haven't looked at all the alternatives very carefully yet). Even using the leading '.' syntax, it would be good to have a specific identifier for the containing package, realized in each module and respected as a special identifier by the package machinery. I like '__pkg__': import __pkg__.sibling # == 'import .sibling' and import __pkg__.__pkg__.uncle # == 'import ..uncle' import __pkg__.__pkg__.uncle.cousin # == 'import ..uncle.cousin' I think the leading '.' dot forms are a lot nicer, but the presence of a module named '__pkg__' would be useful for programmatically mucking navigating the package structure. Ken Manheimer klm@zope.com

On Tue, 23 Dec 2003, Ken Manheimer wrote:

...

This flowed well in my head, and looks natural. However, the suggestion about '.' or '..' referring to the parent looks _ugly and unnatural_.

Let's say '.' was used to spell "parent module", and let's assume that the module 'a.b.c.d' wants to import 'a.b.e'. The absolute import would be spelled:

import a.b.e

... and the relative import would be spelled:

import .....e # .[parent].[parent].e

Yuck! (no offense :)

Truly, yuck. But in my model, the leading '.' dot, itself, stands for the containing package, and '..' stands for the containing package's package, and so forth:

import ..e # [parent-of-parent].e

I can't quite get used to the idea of '.' having two different meanings depending on how many '.'s are found next to each other in an import statement. I believe that the notation above _is_ concise, but _is not_ very intuitive ...

Even using the leading '.' syntax, it would be good to have a specific identifier for the containing package, realized in each module and respected as a special identifier by the package machinery. I like '__pkg__':

import __pkg__.sibling # == 'import .sibling'

and

import __pkg__.__pkg__.uncle # == 'import ..uncle' import __pkg__.__pkg__.uncle.cousin # == 'import ..uncle.cousin'

... while the notation here _is_ intuitive (to a python programmer), but _is not_ concise. I still favor this syntax to the former syntax. I like '__pkg__' better than my initial suggestion ('__parent__'). It's more aesthetically pleasing. :) -- Devin devin@whitebread.org http://www.whitebread.org/