...

Is anyone championing PEP 328? This is planned for inclusion in 2.4, but not much movement has happened. If it's stuck on the decision whether to use multiple dots (one per level up) or a single dot (to indicate searching upwards until found), I'm willing to pronounce that it should be multiple dots.

At least "from . import X" and "from .. import X" are completely clear and more levels up are not likely to occur in practice...

I'm working on producing an edit, but if you just want to Pronounce, that's fine with me. There hasn't been a lot of emotion attached, and nobody has gotten seriously annoyed with the multiple dot idea.

Consider it Pronounced. --Guido van Rossum (home page: http://www.python.org/~guido/)

I guess going grocery shopping removed my voice in this area *wink*.

Actually it was long ago decided that the pronouncement would be either one dot or multiple dots; other ideas have been removed from consideration.

There was a thread on an a VMS-style relative package option that used negative integers for a relative imports. The thread with this line of thought starts with Stephen Horne's post here: http://groups.google.com/groups?q=g:thl3068059620d&dq=&hl=en&lr=&ie=UTF-8&c2coff=1&safe=off&selm=ra4q401p3ld92uthl6l34u8vfqq39567n0%404ax.com

I personally like that it is concise for deeply nested relative imports, and doesn't require counting of dots. We didn't really get any 'elder developers' to comment on that syntax.

OK, since you asked: Yuck. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 2004-03-30 at 17:53, Aahz wrote:

...

Is anyone championing PEP 328? This is planned for inclusion in 2.4, but not much movement has happened. If it's stuck on the decision whether to use multiple dots (one per level up) or a single dot (to indicate searching upwards until found), I'm willing to pronounce that it should be multiple dots.

At least "from . import X" and "from .. import X" are completely clear and more levels up are not likely to occur in practice...

I'm working on producing an edit, but if you just want to Pronounce, that's fine with me. There hasn't been a lot of emotion attached, and nobody has gotten seriously annoyed with the multiple dot idea.

I'd say count me seriously annoyed with the multiple dot idea, but I'm trying not to be annoyed (or annoying) on python-dev today. In general, I'm not a fan of relative imports; I'd rather stick to absolute imports all the time. Given my general outlook, I think a variable number of dots is simply weird and would be hard to read. "Is that three or four dots?" I'd like to think the number of dots would be small, but I've spent a lot of time working on a project with packages hierarchies six levels deep. Jeremy

On Sat, Apr 03, 2004, David Goodger wrote:

I have Aahz's revision ready to check in, but I'm unclear on one point. Is the accepted/pronounced relative import syntax dots only, or can there be a branch too? In other words, is the "Y" here acceptable (where Y is a sibling to the current module's package)?

from ..Y import X

I think the PEP needs a clear statement (and examples) of which syntax **and semantics** have been chosen.

Good point. Here's what I *think* the semantics are; as soon as I get agreement, I'll update the PEP. Given a package layout:: package subpackage1 moduleX moduleY subpackage2 moduleZ moduleA Each leading "." refers to one level of parent. Assuming that the current file is ``moduleX.py``, following are correct usages of the new syntax:: from .moduleY import spam from ..subpackage1 import moduleY from ..subpackage2.moduleZ import eggs from ..moduleA import foo from ...package import bar Actually, I believe that this is the only possible useful semantic; I would welcome clear reasoning for another semantic. Here's a question. Should the following be legal? :: from ...sys import path If not, do we test for it? What kind of error do we raise? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "usenet imitates usenet" --Darkhawk

...

Good point. Here's what I *think* the semantics are; as soon as I get agreement, I'll update the PEP. Given a package layout::

package subpackage1 moduleX moduleY subpackage2 moduleZ moduleA

Each leading "." refers to one level of parent. Assuming that the current file is ``moduleX.py``, following are correct usages of the new syntax::

from .moduleY import spam from ..subpackage1 import moduleY from ..subpackage2.moduleZ import eggs from ..moduleA import foo from ...package import bar

What about names inside package __init__? from .. import bar Is this also valid?

Aahz wrote:

On Mon, Apr 05, 2004, Shane Holloway (IEEE) wrote:

...

...

Aahz:

...

Good point. Here's what I *think* the semantics are; as soon as I get agreement, I'll update the PEP. Given a package layout::

package subpackage1 moduleX moduleY subpackage2 moduleZ moduleA

Each leading "." refers to one level of parent. Assuming that the current file is ``moduleX.py``, following are correct usages of the new syntax::

from .moduleY import spam from ..subpackage1 import moduleY from ..subpackage2.moduleZ import eggs from ..moduleA import foo from ...package import bar

What about names inside package __init__?

from .. import bar

Is this also valid?

Do you mean subpackage1's __init__.py? If so, yes. For most purposes, a package's __init__.py is treated as if it were a module with the name of the package.

Actually, no I meant package's __init__.py -- so that:: from .. import bar as barA from ...package import bar as barB assert barA is barB

Hmmmm... I think I see what you're getting at: how do you access subpackage1 from package's __init__.py using relative imports? Seems to me that you'd have to do ::

from .package import subpackage1

Normally, though, subpackages import from their parents, not the other way around.

I would intuit that one would import subpackage1 from package as:: from . import subpackage1 but that's not explicitly spelled out. Is this correct?

On Mon, Apr 05, 2004 at 01:36:24PM -0400, Aahz wrote:

Your intuition is wrong. From package's __init__.py, ``from .`` refers to package's parent; to get back into the package, you need to use ``from .package``.

That's confusing. I expected '.' to refer to the current package, just as '.' refers to the current directory in most file systems. I think that meaning would be more useful in practice as well. Imagine a package where '.' means the current package: # __init__.py ################################### from .a import A from .b import B ################################################# # a.py ########################################## from .utils import x, y class A: ... ################################################# # b.py ########################################## from .utils import x, z class B: ... ################################################# # utils.py ###################################### def x(): ... def y(): ... def z(): ... ################################################# Notice that the modules that make up the package do not need to know what the package is called. That was one of the major benefits of relative imports that I was looking forward to. Perhaps I misunderstand the PEP. Neil

Neil Schemenauer <nas-python@python.ca>:

On Mon, Apr 05, 2004 at 01:36:24PM -0400, Aahz wrote:

...

Your intuition is wrong. From package's __init__.py, ``from .`` refers to package's parent; to get back into the package, you need to use ``from .package``.

That's confusing. I expected '.' to refer to the current package, just as '.' refers to the current directory in most file systems.

Hmmm. I think people's intuition is likely to differ here depending on whether the module doing the import is a plain module or a package. In a plain module, the only sensible thing for .foo to mean is a module foo in the same package as the importing module. But in a package, it seems to me the most obvious meaning of .foo is a module foo in the package. I would suggest that this is what it *should* mean, since it's going to be very common for a package to refer to submodules of itself, and requiring the name of the package to be included in all such references would defeat one of the purposes of having relative imports, which is to minimise the amount of renaming needed when moving things around in the module hierarchy. Would it be unreasonable to define "." as meaning the current *package*, rather than the current *module* -- i.e. the package itself if the importer is a package, and the containing package if it's a plain module? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Aahz <aahz@pythoncraft.com>:

...

Would it be unreasonable to define "." as meaning the current *package*, rather than the current *module* --

Not unreasonable, but I think it complicates the explanation.

Not if the explanation simply says "'.' refers to the current package", and leaves the reader to deduce what "current package" means when the module is not a package. :-) [1] My intuition is that this will actually cause people less surprises even if they never think about this subtlety before diving in and using the feature. In fact they'll probably be more confused if they *do* think about it, either way... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ Footnotes: [1] A footnote explaining it would be allowed.

I really wish I'd been paying more attention to 328, which I've just read. This proposal seems to break lots and lots and lots of existing code. Maybe I'm reading it wrong. I've written many, many packages that use the form import foo where "foo" is a sibling module in the package. Or from foo import bar where, again, foo is a sibling to the current module. Packages that work like this have been submitted to on-line libraries, printed in books, etc., etc., etc. I really hope I'm reading the PEP wrong! Am I correct in thinking that come Python 2.4 or 2.5, these will just break if the package isn't at the top level (reachable from sys.path)? I was thinking (some months ago) that y'all would come up with an absolute form instead, something like from / import os or import /os or import /email.Message and a relative form like from foo import bar to do fuzzy and imperfect matching of the module (what we have now), or from ./foo import bar to do more explicit matching of siblings, or from ../baz.foo import bar or from -/baz.foo import bar to go up a level. And that the fuzzy and imperfect form would be eventually deprecated, but never removed. Bill

Nope. Absolute imports are the most common form, and we do want to encourage absolute imports whenever possible (because of the conflicts with the stdlib). Relative imports should be "marked".

Wow. This seems amazingly user-hostile. Users (not Python developers) download some two or three year old code to do something from an online library, like the Vaults of Parnassus, and it just breaks because the meaning of "import" has changed. I hope we aren't developing the kind of attitude towards non-developer users that made lispers call them "lusers". Bill

I do wish to caution against assuming the problem is trivial.

I've been down this road myself, and I think I understand the problem. However, whenever I've made changes of this break-itude, I've always done it in a major release. Changing things this way in a minor release just seems hostile to me. Remember that one of the major reasons people use Python (instead of, say, Scheme) is the availability of lots of packages written in Python to do various things. Making those packages buggy is a suboptimal way to win friends and influence people. Bill

On Sat, 2004-04-10 at 12:57, Andrew Koenig wrote:

The trouble is that if you commit yourself to backward compatibility, you wind up with a growing list of decisions for which the default is wrong but you didn't want to break code by changing it. Each time you make such a decision, you make it more difficult the next time to do anything but go for compatibility again. Ultimately, everyone has to remember to override the defaults in lots of everyday circumstances, a mental burden that hurts everyone from novices to experts.

On the other hand, if you don't commit yourself to backward compatibility, you wind up with people downloading old code that breaks because things have changed.

The third possibility is to support all alternatives forever, perhaps with a mechanism such as "from __past__ import foo". That's possible in theory, but if changes include changes in semantics, it may be impossible for old and new to coexist. Moreover, even if they can coexist, testing becomes a nightmare.

It's a trilemma--a choice between three unpleasant alternatives. I don't know the right answer, but I do wish to caution against assuming the problem is trivial.

This is one of the reasons I suggested early on that there be a way to spell -- at the package level by the package developer -- "I'm using absolute imports here". Without that spelling, the package would still use the old semantics, however broken they are <wink>. E.g. if I could put in email/__init__.py something like: from __future__ import i_am_absolutely_resolved_about_my_imports then I'd expect PEP 328 semantics in package email and all subpackages. Older packages wouldn't have this declaration and would operate under the old rules. (I still don't have any need for relative imports. ;) -Barry

What is the rationale given for not allowing: import .moduleY as a relative import statement meaning "from the package containing the module running the import statement, import moduleY"? The PEP suggests that "relative imports will require listing specific names to import". I'm wondering what about "moduleY" in the above isn't specific? FWIW, I read through much of the meanderings of the old threads but never saw a specific damnation-with-cause of this. - C On Sun, 2004-04-11 at 13:23, Aahz wrote:

On Sun, Apr 11, 2004, Barry Warsaw wrote:

...

This is one of the reasons I suggested early on that there be a way to spell -- at the package level by the package developer -- "I'm using absolute imports here". Without that spelling, the package would still use the old semantics, however broken they are <wink>.

E.g. if I could put in email/__init__.py something like:

from __future__ import i_am_absolutely_resolved_about_my_imports

then I'd expect PEP 328 semantics in package email and all subpackages. Older packages wouldn't have this declaration and would operate under the old rules. (I still don't have any need for relative imports. ;)

Anyone else favor having the ``__future__`` directive apply to the package and all its subpackages? (Currently it follows the standard rules of applying only to each module.) Note that this would make it impossible to do what some people want, where a package becomes another package's subpackage, if the new subpackage relies on relative imports under the old rules.

On Mon, Apr 12, 2004, Guido van Rossum wrote:

For all other forms of import, after

import XXX.YYY.ZZZ

then

XXX.YYY.ZZZ

is usable in an expression. (Very different from Java!) But

.moduleY

is not usable in an expression.

(This might be worth adding to the PEP.)

Done! -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups?

On Sun, 2004-04-11 at 13:23, Aahz wrote:

On Sun, Apr 11, 2004, Barry Warsaw wrote:

...

This is one of the reasons I suggested early on that there be a way to spell -- at the package level by the package developer -- "I'm using absolute imports here". Without that spelling, the package would still use the old semantics, however broken they are <wink>.

E.g. if I could put in email/__init__.py something like:

from __future__ import i_am_absolutely_resolved_about_my_imports

then I'd expect PEP 328 semantics in package email and all subpackages. Older packages wouldn't have this declaration and would operate under the old rules. (I still don't have any need for relative imports. ;)

Anyone else favor having the ``__future__`` directive apply to the package and all its subpackages? (Currently it follows the standard rules of applying only to each module.) Note that this would make it impossible to do what some people want, where a package becomes another package's subpackage, if the new subpackage relies on relative imports under the old rules.

Would that be any different than if modules of the subpackage add the __future__ in them? I'm envisioning semantics such that a __future__ in an __init__.py was the same as if that __future__ was explicitly added to every module (i.e. it's a convenience). -Barry

On Mon, Apr 12, 2004, Jeremy Hylton wrote:

On Mon, 2004-04-12 at 10:06, Barry Warsaw wrote:

...

Would that be any different than if modules of the subpackage add the __future__ in them? I'm envisioning semantics such that a __future__ in an __init__.py was the same as if that __future__ was explicitly added to every module (i.e. it's a convenience).

Right. And my point is that there's at least one corner case where this will have definite drawbacks.

I think they belong in every module. A future statement is a compiler gimmick and modules are compiled separately, so each module ought to have its own future statement. If not, the compiler (and human reader) have no idea whether a future statement is in effect. Obviously, the compiler and the reader could learn about some change to import semantics by studying context, but that's extra complexity I'd like to avoid (particular in the case of the compiler).

That's what I expected the reaction to be from others. Barry, if you want your change, lobby Guido. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups?

Aahz <aahz@pythoncraft.com>:

Anyone else favor having the ``__future__`` directive apply to the package and all its subpackages?

-1. Future statements should only apply to the module that contains them. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

On Sat, Apr 10, 2004, Bill Janssen wrote:

Aahz:

...

Nope. Absolute imports are the most common form, and we do want to encourage absolute imports whenever possible (because of the conflicts with the stdlib). Relative imports should be "marked".

Wow. This seems amazingly user-hostile. Users (not Python developers) download some two or three year old code to do something from an online library, like the Vaults of Parnassus, and it just breaks because the meaning of "import" has changed. I hope we aren't developing the kind of attitude towards non-developer users that made lispers call them "lusers".

In addition to Andrew's excellent response, I want to point out that this only affects packages; most of the distributed code that I've seen tends to be in single modules. It also only affects packages that rely on relative imports: as Barry points out, there's a significant set of developers that currently uses only absolute imports because of the tricky semantics for relative imports. Also, by the time imports start to fail, what is currently two or three year-old code will be five or six years old. Anyone who is writing code now will be aware of this issue and should be able to future-proof their code if they want. Nobody should expect to download code that's five or six years old and have it just work. Do you still think this is a "user-hostile" attitude? If yes, you'll need to make your case in more detail. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups?

Bill Janssen wrote:

...

Also, by the time imports start to fail, what is currently two or three year-old code will be five or six years old. Anyone who is writing code now will be aware of this issue and should be able to future-proof their code if they want. Nobody should expect to download code that's five or six years old and have it just work.

I expressed myself poorly; I was thinking of packages written in the future, so they'd still be two or three years old.

"Anyone who is writing code now will be aware..." is an unfortunate example of ivory-tower thinking. Many people who are writing code using a wonderful tool like Python have better things to do than keep up with the vagaries of Python releases. They rightly feel that if they learned Python with 2.2, what they know should be pretty stable at least until Python 3.0 comes out.

Most people in the Python community know that Python doesn't really have "minor releases" other than bugfix releases. Just like Linux and Perl, the choice to switch to a big version number is more marketing than anything. (and it says something about our marketing saavy that we've done it exactly once)

... They don't think that in a minor release, Python import semantics will change out from under them. Every few years, when they find some time, they will check out some of the new features, like generators, that the developers find so delightful. But in between those times, they won't be clueful.

It really just boils down to a question of how much time you give people. First you try to publicize the change via books, tutorials, mailing lists etc. Then you add an annoying warning. Then eventually you break the old code. If you move slow enough then I claim you can make an arbitrarily deep and pervasive change without significant user pain. For example consider the shift over the years from K&R C to ANSI C to C++. It wasn't totally painful but it was never so painful as to make people switch languages on the basis of backwards incompatibility pain. And there were some pretty major changes in there...the deprecation of implicit casting of void * being one that pops to mind.

One of the attractive features of Python has always been that 5-year-old code *does* just work. That's because the design was largely right from the start, and the designers have been careful to keep things backwards-compatible.

5-year-old code OFTEN just works. But not always. Each version of Python has a document that describes incompatibilities. http://www.python.org/1.6/ http://www.python.org/2.2.3/bugs.html e.g. "The special attributes __members__ and __methods__ are no longer supported (for most built-in types). Use the new and improved dir() function instead." Admittedly this relative import change is a big one compared to those from the past. My opinion is that that just means it should be done a very long time after the version with the warning ships. My opinion is that the process is correct and it is just a question of whether the timelines are generous enough or not. Somewhere between "Python 2.6" and "never" there must be a point where people have had enough time to change over their code after getting nagged by the DeprecationWarning. Paul Prescod

Aahz <aahz@pythoncraft.com>:

Nope. Absolute imports are the most common form

I don't agree with that. In every major Python application I've written, the vast majority of imports are from other modules of the application, which are most sensibly referenced relatively. I, too, am quite concerned about the imminent breaking of every existing relative import, apparently because Guido has an ideological objection to relative imports and wants to punish us for having used them so freely over the years. If he goes ahead with this, I suggest we hack his unix system and remove the ability to use relative pathnames. Let's see how he likes using that. :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Bill Janssen <janssen@parc.com> writes:

I really wish I'd been paying more attention to 328, which I've just read. This proposal seems to break lots and lots and lots of existing code.

Lots of *your* code, maybe... I think you might be one of the few people who embraced packages as soon as they were available.

Maybe I'm reading it wrong.

I've written many, many packages that use the form

import foo

where "foo" is a sibling module in the package. Or

from foo import bar

where, again, foo is a sibling to the current module.

Nevertheless, do you agree or not that it was a mistake that relative and absolute imports are spelt the same? I you do, then surely the longer we leave it, the worse the problem gets... [...]

I was thinking (some months ago) that y'all would come up with an absolute form instead, something like

Then you really haven't been paying attention! Pretty much *everyone* who has contributed to the discussion, and presumably plenty who haven't (like me), has assumed that absolute imports are more common and should the default (certainly, all the imports in my own code are absolute...). Cheers, mwh --

With Python you can start a thread, but you can't stop it. Sorry. You'll have to wait until reaches the end of execution. So, just the same as c.l.py, then? -- Cliff Wells & Steve Holden, comp.lang.python

David Goodger <goodger@python.org> writes:

...

In other words, is the "Y" here acceptable (where Y is a sibling to the current module's package)?

from ..Y import X

David Abrahams wrote:

Wouldn't that have to be

from ...Y import X ^-------------separator, as in A.B ??

Aha! My confusion is confirmed. I hadn't thought of requiring a separator "."; that seems like overloading of meaning. I understood the syntax as Aahz describes it, so a single leading "." means "current package" (i.e., sibling of current module), ".." means "parent package" or "up one level", etc.

I'm thinking some other syntax is needed to represent upward travel in the hierarchy, e.g.

from ^.Y import X

That has a certain elegance to it. So ".Y" would mean import from current package, "^.Y" would mean import from parent package, "^^.Y" from parent's parent package, etc. I don't want to reopen debate needlessly, but the overloading of the meaning of "." *is* a bit confusing here. Whatever makes sense to Guido will eventually make sense to me, so I'm not worried. -- David Goodger

On Sat, Apr 03, 2004, David Abrahams wrote:

I'm thinking some other syntax is needed to represent upward travel in the hierarchy, e.g.

from ^.Y import X from -.Y import X

<ducks>

That's been suggested, and it'll show up in the revised PEP as soon as it's pushed out to python.org. I'm not Uncle Timmy, but I'll channel Guido: no chance. I don't particularly like dots-only mechanism, but I have to agree that it works well enough for one-to-three levels. More than that is a strong indication of a design that needs reworking. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "usenet imitates usenet" --Darkhawk

Guido van Rossum <guido@python.org> writes:

...

...

from ..Y import X

Wouldn't that have to be

from ...Y import X ^-------------separator, as in A.B

NO, NO, NO! Each dot is one uplevel. Exactly like Mac OS 9 paths do with ':'. (Read macpath.py.)

That's a fine convention; I'm familiar with MacOS paths. The fact that David Goodger and I were both confused might indicate that the proposed syntax might not be as intuitive as you'd like, but I don't think it'll take long to get used to. -- Dave Abrahams Boost Consulting www.boost-consulting.com

Guido van Rossum <guido@python.org> writes:

...

...

NO, NO, NO! Each dot is one uplevel. Exactly like Mac OS 9 paths do with ':'. (Read macpath.py.)

That's a fine convention; I'm familiar with MacOS paths. The fact that David Goodger and I were both confused might indicate that the proposed syntax might not be as intuitive as you'd like, but I don't think it'll take long to get used to.

I think the only problem was that it might not have been explained clearly enough.

Sure, that would handle it. Fails to meet a high standard set by other elements of Python that can be grasped immediately, without any explanation, but of course not everything can be that simple. -- Dave Abrahams Boost Consulting www.boost-consulting.com